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PREFACE 


This  report  describes  the  PACER  SHARE  Productivity  and  Personnel 
Management  Demonstration  and  the  plan  that  has  been  developed  to 
evaluate  it.  The  report  also  presents  statistical  results  concerning  the 
quality  of  work  life,  organizational  flexibility,  work  quality,  and  cost 
savings  during  the  baseline  period  prior  to  the  demonstration  and 
through  the  demonstration’s  first  two  years. 

PACER  SHARE  is  a  five-year  demonstration  being  conducted  at  the 
Directorate  of  Distribution  (DS)  within  the  Sacramento  Air  Logistics 
Center  (SM-ALC)  under  the  legal  authority  of  the  Office  of  Personnel 
Management.  Its  purpose  is  to  determine  whether  several  changes  in 
federal  civil  service  practices  being  tried  on  an  experimental  basis 
will  improve  organizational  productivity,  flexibility,  and  quality  of 
work  life  while  sustaining  (or  improving)  the  quality  and  timeliness  of 
work  and  the  capability  to  mobilize  during  emergency  or  wartime. 
The  DSs  at  the  four  remaining  ALCs  (which  perform  similar  func¬ 
tions)  serve  as  the  comparison  sites.  The  demonstration  formally  be¬ 
gan  in  February  1988  after  several  years  of  planning.  If  effective,  the 
interventions  will  subsequently  be  considered  for  wider  application. 

RAND  is  responsible  for  designing  and  cariying  out  the  external 
evaluation  of  the  project’s  results.  These  results  should  be  of  interest 
to  the  wide  audience  concerned  with  the  improvement  of  the  work¬ 
place  within  the  public  sector. 

The  study  is  funded  by  the  U.S.  Air  Force  through  a  special  arrange¬ 
ment  with  the  Assistant  Secretary  of  Defense  (Force  Management 
and  Personnel),  the  research  sponsor.  It  is  being  carried  out  by  the 
Defense  Manpower  Research  Center,  a  component  uf  RAND’s 
National  Defense  Research  Institute,  a  federally  funded  research  and 
development  center  sponsored  by  the  Office  of  the  Secretary  of 
Defense  and  the  Joint  Staff. 


SUMMARY 


The  PACER  SHARE  Demonstration  Project  is  a  five-year  federal  civil 
service  demonstration  being  conducted  within  the  Directorate  of 
Distribution  (DS)  at  the  Sacramento  Air  Logistics  Center  (ALC),  un¬ 
der  the  legal  authority  of  the  Office  of  Personnel  Management.  In 
broad  terms,  the  goals  of  PACER  SHARE  are  to  increase  the  flexibil¬ 
ity  of  the  organization  to  respond  to  changes  in  workload;  to  enrich 
the  quality  of  work  life;  to  maintain  the  quality  and  timeliness  of 
work  as  these  changes  are  being  brought  about  and,  in  the  long  run, 
to  make  quality  and  timeliness  even  better;  and  to  enhance  productiv¬ 
ity.  The  demonstration  is  designed  to  attain  these  objectives  through 
several  innovations  in  personnel  practices  and  through  productivity 
gainsharing,  which  returns  one-half  of  cost  savings  to  the  work  force. 
The  personnel  system  changes  include  job  series  consolidation;  re¬ 
vised  base  pay  determination,  including  pay  banding  and  elimination 
of  individual  performance  appraisal;  supervisory  grading  criteria 
changes  that  emhasize  job  responsibilities  and  deemphasize  number 
of  subordinates;  and  Demonstration  On-Call  hiring  authority,  which 
provides  for  rapid  employee  release  and  recall.  Results  in  achieving 
PACER  share’s  goals  are  reported  through  spring  1990,  after  two 
years  of  the  demonstration. 

The  improvement  of  organizational  flexibility  and  quality  of  work  life 
is  measured  through  a  series  of  survey  and  hard  data  (nonsurvey) 
measures.  First,  attitude  changes  among  DS  workers  are  measured 
in  extensive  annual  surveys  of  the  work  forces  at  Sacramento  and  the 
other  ALCs.  Second,  a  battery  of  hard  data  measures  addresses  other 
changes.  These  measures  are  drawn  primarily  from  the  automated 
civilian  personnel  system.  For  example,  we  look  at  supervis- r 
staffing  and  instances  of  multiple  skill  training  to  assess  organiza¬ 
tional  flexibility  and  at  turnover  rates  to  address  quality  of  work  life. 

Quality  and  timeliness  of  work  are  addressed  through  an  evaluation 
of  command-directed  measures,  including  data  on  error  rates  main¬ 
tained  by  the  Quality  Division  of  DS,  reports  of  discrepancies,  and 
measures  of  timeliness  and  support  maintained  by  tne  Management 
Division.  Finally,  we  assess  changes  in  productivity  by  measures  of 
personnel  costs  relative  to  output.  The  methou  we  use  is  based  on 
statistical  estimates  of  the  changes  over  time  in  personnel  costs  at 
Sacramento  and  the  other  ALCs,  controlling  for  changes  in  output. 


VI 


This  method  produces  somewhat  different  results  from  the  method 
used  by  DS  at  Sacramento  to  determine  gainshares. 

In  each  area  our  strategy  is  to  identify  changes  that  occur  under 
PACER  SHARE  and  to  distinguish  them  from  changes  occurring  in 
DS  at  the  other  ALCs  during  the  same  time  period.  This  means  es¬ 
tablishing  predemonstration  (baseline)  levels  for  all  measures  at 
Sacramento  and  the  comparison  sites,  reassessing  the  measures  an¬ 
nually  to  determine  changes,  and  comparing  the  extent  of  change  at 
Sacramento  with  that  at  the  other  ALCs  so  as  to  identify  the  effects 
unique  to  PACER  SHARE;  this  amounts  to  identifying  the  difference 
in  the  amount  of  change  occurring  at  Sacramento  and  at  the  other 
ALCs,  taken  as  a  whole. 

ORGANIZATIONAL  FLEXIBILITY  AND  QUALITY  OF  WORK 
LIFE:  ATTITUDES  TOWARD  WORK 

We  will  begin  by  reviewing  changes  in  attitudes  toward  the  work  en¬ 
vironment  through  the  second  year  of  the  demonstration.  In  distinc¬ 
tion  to  the  results  at  the  end  of  year  one,  which  provided  only  limited 
evidence  of  improvement,  a  broad  series  of  responses  indicates  that 
perceptions  of  overall  working  conditions  had  improved  significantly 
by  the  end  of  year  two.  Satisfaction  with  supervision  and  co-worker 
interactions,  overall  work  satisfaction  (the  meaningfulness  of  the  job), 
and  other  work  environment  perceptions  (trust  in  management,  con¬ 
trol  of  work,  training  opportunities,  organizational  involvement)  im¬ 
proved  significantly  at  Sacramento  by  year  two  as  compared  with  the 
other  ALCs,  where  little  systematic  change  occurred  during  the  two- 
year  period.  In  contrast,  and  consistent  with  the  year-one  results, 
satisfaction  with  pay  and,  particularly,  with  the  connection  between 
job  performance  and  compensation  or  advancement  worsened  signifi¬ 
cantly  as  compared  to  baseline.  Also  as  true  for  year  one,  further 
analysis  indicates  that  the  pay  and  advancement  concerns  may  have 
depressed  nonpay-related  attitudes  such  as  those  concerning  the 
three  areas  mentioned  previously.  The  implication  is  that  dissatis¬ 
faction  with  pay  and  advancement  under  the  job  series  consolidation 
and  revised  base  pay  determination  innovations  must  be  addressed 
before  we  can  be  certain  that  improvement  in  nonpay  areas  has  been 
observed  fully. 

In  addition  to  their  relation  to  quality  of  work  life,  attitude  changes 
can  be  addressed  in  terms  of  their  consistency  with  the  goal  of  in¬ 
creasing  organizational  flexibility  under  PACER  SHARE.  In  contrast 
to  year  one,  when  changes  were  limited,  year-two  attitude  changes 
generally  supported  this  goal.  For  example,  perceived  staffing  flexi- 
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bility,  training  opportunities,  job  competition  fairness,  supervisory 
grading  criteria  fairness,  gainsharing  of  organizational  cost  savings, 
and  Personnel  Office  helpfulness  increased.  Other  perceptions 
showed  no  change  relative  to  the  comparison  ALCs:  blue-collar  ver¬ 
sus  white-collar  pay  equity,  staffing  needs,  job  classification  satisfac¬ 
tion,  and  career  opportunities. 

Finally,  additional  attitude  questions  showed  increases  in  perceived 
information  exchange  in  accomplishing  day-to-day  work,  usefulness  of 
quality  circle  participation,  and  emphasis  of  team-building  concepts 
in  day-to-day  operations.  These  changes  are  consistent  with  both  bet¬ 
ter  quality  of  work  life  and  improved  work  quality.  Many  of  the 
changes  (especially  those  for  team  building)  were  large  and  highly 
significant. 

ORGANIZATIONAL  FLEXIBILITY  AND  QUALITY  OF  WORK 
LIFE;  PERSONNEL  MEASURES 

The  results  from  the  automated  personnel  system  provide  greater 
support  for  PACER  SHARE  than  was  true  at  the  end  of  year  one. 
Our  analysis  indicates  general  improvement  from  baseline  levels  at 
Sacramento  relative  to  changes  at  the  other  ALCs  during  the  two- 
3-ear  time  period,  Consistent  with  both  increased  organizational  flex¬ 
ibility  (the  provision  of  greater  salary  potential  through  banding  and 
annual  pay  increases)  and  quality  of  work  life  (resulting  from  higher 
pay),  average  salary  growth  in  DS  during  the  two-year  period  was 
greater  at  Sacramento  than  at  the  other  ALCs.  The  change  varied  by 
pay  band,  but  wages  in  the  largest  bands  increased  significantly  rela¬ 
tive  to  the  comparison  sites.  Among  continuing  employees,  salaries 
grew  at  a  rate  equal  to  or  greater  than  those  for  comparable  pay 
bands  at  the  other  ALCs.  And,  there  was  no  evidence  of  pay  inversion 
between  supervisors  and  nonsupervisors. 

These  results  are  noteworthy  in  showing  that  most  employees  fared 
as  well  or  better  financially  under  job  series  consolidation  and  pay 
banding  than  did  their  counterparts  at  the  other  ALCs — operating 
under  the  traditional  system — despite  the  worsening  of  pay-related 
attitudes.  In  other  words,  pay  dissatisfaction  appears  to  be  a  matter 
of  perception.  Evidence  suggests  that  the  cost  of  living  rose  more 
rapidly  in  the  Sacramento  area  than  at  the  comparison  sites,  offset¬ 
ting  salary  growth;  this  may  have  contributed  to  the  worsening  of  pay 
satisfaction.  The  worsening  of  attitudes  concerning  the  link  between 
job  performance  and  compensation  or  advancement  is  more  likely  tied 
to  the  changes  in  annual  salary  increases  and  promotion  practices 
under  job  series  consolidation  and  pay  banding.  It  also  should  be 


noted  that  the  salary  growth  was  offset  by  a  reduction  in  the  size  of 
the  work  force.  As  discussed  below,  overall  labor  costs  did  not  in¬ 
crease. 

Three  other  measures  of  organizational  flexibility  concerned  the 
number  of  skills  per  employee,  the  percentage  of  supervisors,  and 
white-  to  blue-collar  crossovers.  Consistent  with  the  goals  of  expand¬ 
ing  training  and  increasing  organizational  flexibility,  the  average 
number  of  skills  grew  significantly  more  at  Sacramento  during  the 
two-year  period;  that  is,  it  appears  more  skill  training  was  provided, 
amounting  to  about  one  more  skill  for  every  three  workers  than  at  the 
other  ALCs.  In  contrast,  the  percentage  of  supervisors  remained  un¬ 
changed,  both  overall  and  within  divisions.  On  one  hand,  this  sug¬ 
gests  that  the  flexibility  provided  by  changes  in  supervisory  grading 
criteria  has  not  yet  been  fully  utilized  (although  it  must  be  recognized 
that  changes  such  as  movement  of  supervisors  below  the  division 
level  could  have  occurred  under  the  new  criteria).  On  the  other,  it  in¬ 
dicates  that  the  removal  of  the  number  and  grades  of  subordinates  as 
criteria  for  establishing  supervisory  positions  did  not  result  in  prolif¬ 
eration  of  supervisory  positions  at  Sacramento.  'WTiite-  to  blue-collar 
crossover  increased  at  the  other  ALCs  during  year  two.  Despite  re¬ 
vised  base  pay  determination  procedures — which  seek  in  part  to  re¬ 
duce  the  economic  incentive  for  crossovers — white-  to  blue-collar 
crossovers  increased  comparably  at  Sacramento. 

Finally,  two  measures  of  turnover  were  examined  as  they  bear  on 
quality  of  work  life  and  hiring  practices.  It  was  believed  that  the  per¬ 
centage  of  career  employees  might  decrease  if  senior  personnel  were 
unhappy  with  PACER  SHARE.  In  fact,  the  results  show  that  the  per¬ 
centage  of  career  employees  grew  significantly  at  Sacramento  relative 
to  that  at  the  comparison  sites  during  the  two-year  period.  In  addi¬ 
tion  to  possible  changes  in  quality  of  work  life,  the  increase  reflects 
fewer  hiring  actions  taken  at  Sacramento  to  fill  vacated  positions. 
Also  consistent  with  greater  quality  of  work  life,  the  turnover  rate  de¬ 
creased  significantly  at  Sacramento.  It  thus  became  similar  to  that  at 
the  other  ALCs,  rather  than  maintaining  its  historically  higher  rate. 

WORK  QUALITY 

Our  analysis  of  work  quality  focused  on  command-directed  measures 
of  error  rates  and  reports  of  discrepancies  (RODs)  maintained  by  the 
Quality  Division  and  measures  of  timeliness  and  support  maintained 
by  the  Management  Division.  The  findings  resemble  those  of  year 
one.  Year-two  error  rates  for  controlled  exceptions,  packing  process, 
receiving  inspection,  tailgate  accuracy,  location  audit  program  survey. 


IX 


and  physical  count  of  noncontrolled  items  showed  no  overall  pattern 
of  change  from  baseline,  at  which  time  rates  at  Sacramento  were  su¬ 
perior  to  those  at  the  comparison  sites.  Similarly,  RODs  showed  no 
overall  pattern  of  change  from  baseline,  remaining  lower  at 
Sacramento.  Finally,  there  was  a  relative  decline  in  timeliness  and 
support — as  assessed  by  receiving  document  posting,  binning,  high- 
priority  requisitions,  and  denials — relative  to  the  other  ALCs.  The 
two  largest  changes  occurred  for  binning  timeliness  and  requisitions, 
and  are  believed  by  Sacramento  to  be  at  least  partially  attributable  to 
Automated  Warehouse  System  implementation  and  management 
emphases  (such  as  F-15  support).  Future  analyses  will  attempt  to 
shed  additional  light  on  these  findings. 

COST  SAVINGS 

With  respect  to  labor  cost  savings,  PACER  SF*  .as  yielded  a 
steady  stream  of  gainshare  payments  over  the  i  quarters  beginning 
January  1989,  following  several  quarters  without  gainshares.  Indeed, 
our  analysis  suggests  that  had  the  gainsharing  formula  included  an 
adjustment  for  change  in  output  level  between  baseline  and  demon¬ 
stration  period,  Sacramento’s  gainshares  would  have  been  still 
higher.  In  any  case,  the  gainshare  payments  imply  that  labor  cost 
has  been  below  its  baseline  level.  Gainshare  payments,  however,  are 
not  an  adequate  basis  for  judging  whether  PACER  SHARE  has 
brought  a  systematic  improvement  in  Sacramento’s  productivity.  For 
that,  we  rely  on  a  multivariate  model  of  labor  cost  as  a  function  of 
time  and  output,  which  we  estimate  separately  for  Sacramento  and 
the  other  ALCs,  for  both  the  baseline  and  demonstration  periods.  The 
estimated  model  forms  the  basis  of  a  series  of  hypothesis  tests.  Of 
most  importance,  the  tests  show  that  Sacramento’s  labor  cost  under 
PACER  SHARE  was  not  statistically  different  from  that  prior  to 
PACER  SHARE.  Even  though  gainshares  were  paid,  costs  fell  within 
the  range  expected  from  Sacramento’s  pre-PACER  SHARE 
performance.  Furthermore,  we  tested  Sacramento’s  cost  relative  to 
that  of  other  ALCs,  whose  behavior  reflects  what  might  have  been 
expected  for  Sacramento  had  there  been  no  PACER  SHARE.  We 
found  no  statistically  significant  evidence  that  Sacramento  had 
reduced  its  cost  relative  to  that  of  its  peers. 

The  potential  for  cost  savings  remains  and  could  improve  as  PACER 
SHARE  matures.  More  workers,  for  instance,  will  become  trained  in 
multiple  skills  and  therefore  qualified  for  assignment  to  a  wide  vari¬ 
ety  of  tasks  as  they  arise.  Still,  the  potential  for  cost  improvement 


could  be  affected  in  the  future  by  factors  outside  DS  control  such  as 
Desert  Storm,  reorganization,  and  absorption  by  the  Defense 
Logistics  Agency. 
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1.  INTRODUCTION 


This  report  evaluates  the  first  two  years  of  the  PACER  SHARE 
Productivity  and  Personnel  Management  Demonstration.^  PACER 
SHARE  is  a  five-year  demonstration  initiated  by  the  Directorate  of 
Distribution  (DS)  at  the  Sacramento  Air  Logistics  Center  (ALC)  un¬ 
der  the  authority  of  Title  VI  of  the  Civil  Service  Reform  Act.  That  ti¬ 
tle  allows  the  U.S.  Office  of  Personnel  Management  (0PM)  to  waive 
federal  civil  service  regulations  on  an  experimental  basis  to  deter¬ 
mine  whether  alternative  procedures  improve  public  personnel  man¬ 
agement.  (Appendix  A  describes  other  demonstration  projects  autho¬ 
rized  by  0PM.)2  After  several  years  of  planning,  the  project  was 
formally  initiated  in  February  1988. 

The  demonstration  is  designed  to  determine  whether  certain  innova¬ 
tions  will  improve  organizational  productivity,  flexibility,  and  quality 
of  work  life  while  sustaining  the  quality  and  timeliness  of  work  and 
the  capability  of  mobilizing  during  emergencies  or  wartime.  If  suc¬ 
cessful,  the  interventions  constituting  the  project  will  be  considered 
for  wider  application  in  the  federal  sector.  RAND  is  the  external 
evaluator. 

This  section  provides  the  background  on  PACER  SHARE  necessary 
for  understanding  the  methods  and  results  presented  in  the  rest  of 
the  report.  The  material  is  largely  drawn  from  OPM’s  announcement 
of  PACER  SHARE  in  the  Federal  Register^  and  provides  the  justifi¬ 
cation  for  the  demonstration  contained  therein.  This  section  and  the 
one  on  methods  largely  repeat  information  given  in  the  PACER 
SHARE  baseline  report.  Readers  familiar  with  that  report  may  wish 
to  proceed  directly  to  the  discussion  of  results,  which  begins  in  Sec.  3. 

RATIONALE 

Broadly  speaking,  the  demonstration  grew  out  of  criticism  of  the  fed¬ 
eral  civil  service  system  (criticism  that  its  system  of  classification  and 
compensation,  staffing,  incentives,  and  performance  appraisal  im¬ 
pedes  efficiency).  The  federal  job  classification  system,  for  example,  is 


IPor  baseline  and  first-year  findings,  sec  R-3753-FMP,  1990,  and  R-3943-FMP, 
1991. 

^Appendices  B  and  C  are  located  at  the  back  of  this  volume.  Other  appendix 
citations  in  this  report  refer  to  the  companion  Note,  N-3404-FMP,  1991. 

^Federal  Register,  Vol.  52,  No.  224,  November  20,  1987,  pp.  44782-44810. 
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complex.  It  divides  work  into  a  large  number  of  small  pieces.  The 
General  Schedule  (GS)  for  white-collar  work  defines  440  jobs  and  is 
further  divided  into  18  grades,  or  levels  of  difficulty.  The  Federal 
Wage  System  (FWS)  for  blue-collar  work  defines  330  jobs  in  15 
grades.  The  system’s  emphasis  on  classification  accuracy  encourages 
the  design  of  narrow  jobs.  Furthermore,  managers  have  little  control 
over  the  pay  rates  of  their  employees,  classified  as  they  are  within 
narrow  grades  with  limited  room  for  within-grade  increases.  This  is 
an  especially  important  problem  for  white-collar  employees,  whose 
pay  does  not  reflect  local  market  conditions.  Pay  raises  are  thus  often 
effected  by  reclassifying  jobs.  As  a  result,  managers  and  personnel 
specialists  devote  much  time  to  writing  the  frequently  lengthy  posi¬ 
tion  descriptions  required  to  justify  narrow  grade  and  series  distinc¬ 
tions. 

The  rigid  specialization  arising  from  narrow  job  classification  can 
contribute  to  the  employment  of  more  personnel  than  may  be  re¬ 
quired.  Narrow  and  restrictive  qualifications  interfere  with  a  super¬ 
visor’s  ability  to  assign  work  to  employees  as  needed.  Managers  who 
try  to  establish  broader,  generalist  jobs  are  likely  to  have  trouble  get¬ 
ting  them  classified.  At  the  supervisory  level,  grading  criteria  (such 
as  number  of  subordinates  supervised)  can  encourage  “empire  build¬ 
ing”  and  the  creation  of  additional  layers  of  supervision. 

Similar  concerns  apply  to  stalling.  Paperwork  is  required  when  an 
employee  is  assigned  to  different  duties,  whether  the  action  is  perma¬ 
nent  or  temporary.  This  documentation  consumes  time  and  personnel 
resources,  interfering  with  other  needed  actions.  Reduction-in-force 
(RIF)  procedures  for  decreasing  the  size  of  the  work  force  are  also 
costly  and  time-consuming. 

The  incentive  awards  system  provides  limited  tools  for  motivating  the 
entire  work  force.  Awards  generally  go  to  only  a  small  number  of 
employees  and  are  not  perceived  as  widely  available. 

Performance  appraisal  requires  the  development  of  performance 
plans  containing  elements  and  standards  for  every  employee. 
Performance  is  then  evaluated  using  five  rating  levels.  Setting  and 
measuring  performance  at  five  levels  for  every  job,  regardless  of 
complexity,  increase  paperwork  and  consume  time.  Moreover,  ap¬ 
praisal  does  not  address  measurement  of  aggregate  productivity  and 
work  quality,  which  are  more  important  to  the  Directorate  of 
Distribution  than  individual  performance. 
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PROJECT  GOALS 

The  PACER  SHARE  demonstration  addresses  these  criticisms  with  a 
set  of  five  interventions  put  forward  by  the  Directorate,  described  be¬ 
low.  Overall,  the  objectives  of  PACER  SHARE  are  to: 

•  Increase  organizational  productivity  by  improving  incentives  and 
training  to  help  employees  work  more  effectively  and  encourage 
them  to  originate  ideas  on  improving  efficiency. 

•  Increase  organizational  flexibility  in  making  job  assignments  and 
dealing  with  fluctuations  in  workload, 

•  Enrich  the  quality  of  work  life  by  creating  a  work  environment  in 
which  individual  and  organizational  goals  are  compatible,  opportu¬ 
nities  for  individuals  to  work  on  a  variety  of  jobs  are  realized,  and 
training  opportunities  are  expanded. 

•  Preserve  or  improve  the  quality  and  timeliness  of  work  through 
quality  circles, team  building,  and  statistical  process  control.® 

Achieving  these  goals  requires  the  adoption  of  a  new  management 
philosophy  that  encourages  greater  involvement  of  all  employees  in 
the  problems  and  challenges  faced  by  their  organization.  Part  of  this 
philosophy  entails  building  a  sense  of  “corporate  identity”  in  every  di¬ 
rectorate  member.  This  philosophy,  adopted  by  the  Japanese  in  the 
early  1950s,  was  more  recently  set  out  by  Doming  (1987). 

The  demonstration  aims  to  promote  corporate  identity  by  involving 
employees  directly  in  improving  their  organization’s  productivity  and 
quality  of  work  life.  Productivity  (during  peacetime  or  wartime) 
would  be  increased  by  having  a  more  versatile  work  force,  more  flex¬ 
ibility  in  making  job  assignments,  greater  ease  in  dealing  with  fluctu¬ 
ations  in  workload,  and  more  latitude  for  establishing  supervisory  po¬ 
sitions.  At  the  same  time,  there  would  be  a  continuation  of  effort  to 
improve  work  quality  and  timeliness  by  means  of  statistical  process 
control  and  by  encouraging  worker  participation  in  diagnosing  and 
correcting  problems  and  proposing  new  solutions. 

PACER  SHARE  is  the  first  OPM-authorized  demonstration  to  be  con¬ 
ducted  in  a  unionized  environment.  In  this  era  of  resource  scarcity, 
labor-management  cooperation  is  important  if  new  methods  are  to  be 


^Quality  circles  are  small  groups  that  convene  regularly  to  discuss  work  problems 
and  means  of  improvement. 

^Statistical  process  control  uses  statistical  piotocols  to  help  identify  work  processes 
or  areas  subject  to  undesirable  variations  in  quality  and  to  bring  such  variations  under 
control  by  correcting  problems  with  the  work  process. 
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developed  for  “doing  more  with  less.”  The  demonstration  rests  on  the 
hypothesis  that  most  employees  want  to  work  effectively,  and  if  given 
proper  incentives  and  encouraged  to  participate,  employees  will  come 
up  with  more  productive  ways  of  doing  their  work. 

DEMONSTRATION  SITE  AND  POPULATION 

The  Directorate  of  Distribution  at  the  Sacramento  ALC  operates  un¬ 
der  the  authority  of  the  Air  Force  Logistics  Command.  The  direc¬ 
torate  consists  of  warehouses  and  related  facilities  for  receiving, 
storing,  and  shipping  materiel  in  support  of  U.S.  Air  Force  operations 
around  the  world.  Such  materiel  includes  spare  parts,  small  arms, 
uniforms,  and  food  rations.  Items  are  received  by  the  directorate 
from  manufacturers  or  from  Air  Force  facilities  where  they  are  not 
needed.  They  are  logged  in  and  stored  until  they  are  needed  by  an¬ 
other  Air  Force  facility,  when  they  are  packed  and  shipped.  Five 
other  directorates  operate  at  the  Sacramento  ALC: 

•  Maintenance  repairs  and  modifies  weapon  systems,  performs  non¬ 
destructive  testing  of  parts,  and  manufactures  parts; 

•  Materiel  Management  determines  stock  levels  of  weapon  systems, 
parts,  and  equipment;  decides  whether  to  repair  or  buy  new 
equipment;  and  initiates  R&D  contracts  to  improve  weapon  system 
reliability  and  performance; 

•  Contracting  and  Manufacturing  locates  sources  for  needed  parts 
and  services  and  manages  procurement  actions; 

•  Communications  and  Computer  Systems  operates  and  maintains 
mainframe  computer  resources  for  the  ALC,  develops  and  main¬ 
tains  data  systems,  and  provides  programming  and  systems 
analysis  support; 

•  Competition  Advocacy  finds  or  develops  additional  sources  of  sup¬ 
ply  in  the  private  sector  and  researches  proposed  procurement 
actions  for  possible  overpricing. 

At  baseline,  the  DS  was  staffed  by  approximately  1800  civilians,  wbo 
are  participating  in  the  demonstration,  and  120  military  personnel, 
who  are  not.® 


^Military  personnel  are  not  participating  in  the  demonstration  because  they  are 
prohibited  by  law  from  participating  in  gainsharing  plans  (one  of  the  five  interventions) 
and  because  the  remaining  interventions  concern  changes  in  the  civilian  personnel 
system. 
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A  salient  feature  of  this  demonstration  distinguishing  it  from  previ¬ 
ous  ones  is  the  involvement  of  large  numbers  of  both  blue-  and  white- 
collar  and  both  line  and  management  personnel.  The  staff  is  divided 
evenly  between  General  Schedule  and  Federal  Wage  System  (see 
Table  1).  Another  important  feature  is  the  involvement  of  labor 
unions  in  the  project.  At  baseline,  about  20  percent  of  DS  employees 
were  represented  by  a  union.'^  Of  those,  87  percent  belonged  to  the 
American  Federation  of  Government  Employees;  the  remainder  were 
members  of  the  Technical  Skills  Association  or  the  Engineer  and 
Scientist  Association. 


Table  1 

Participating  Employees  by  Pay  Schedule® 
(Directorate  of  Distribution,  Sacramento  Air 
Logistics  Center) 


Pay  Schedule  and 
Supervisory  Status^ 

Percentage  of 
Work  Force 

Number  of 
Employees 

Nonsupervisory  GS 

44.2 

808 

Supervisory  GS  and  PMRS 

5.7 

105 

Nonsupervisory  FWS 

45.8 

838 

Supervisory  FWS 

4.3 

78 

Total 

100.0 

1829 

“The  figures  reflect  work  force  status  as  of  February  1988. 
^GS  is  General  Schedule  (white  collar);  PMRS  is  Per¬ 
formance  Management  and  Recognition  System;  FWS  is 
Federal  Wage  System  (blue  collar). 


INTERVENTIONS 

Achieving  the  project  goals  requires  improvements  in  organizational 
flexibility  and  a  shift  from  individually  oriented  to  organizationally 
based  incentives.  Accordingly,  five  interventions  were  designed: 

•  Job  series  consolidation 

•  Revised  base  pay  determination: 

-  Pay  banding 

-  Eliminating  individual  annual  performance  ratings 

•  Revised  supervisory  grading  criteria 


^According  to  a  survey  of  the  work  force  (described  below). 
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•  Revised  hiring/retention  criteria 

•  Productivity  gainsharing 


Job  Series  Consolidation 

Under  PACER  SHARE,  DS’s  66  job  series  were  consolidated  into  six 
broad  “processes.”  A  process  was  defined  as  “the  progressive  and  in¬ 
terdependent  arrangement  of  events,  machines,  methods,  and  re¬ 
sources  that  produce  a  good  or  service.”  Jobs  that  contribute  to  the 
same  goal  were  assigned  to  the  same  process. 

The  demonstration  consolidated  27  blue-collar  series  into  two  pro¬ 
cesses: 

•  Material  handling  process  encompasses  physically  receiving,  exam¬ 
ining,  packing,  moving,  storing,  and  issuing  items. 

•  Facilities  and  equipment  maintenance  covers  physically  maintain¬ 
ing  and  repairing  material  processing  equipment  and  facilities. 
This  is  divided  into  seven  subprocesses  (electronic,  electrical,  metal 
working,  painting,  carpentry,  industrial-equipment  repair,  and 
mobile-equipment  repair). 

Second,  39  nonsupervisory  white-collar  series  were  consolidated  into 
three  processes. 

•  Distribution  process  covers  custody  and  transportation  transac¬ 
tions. 

•  Management  operations  process  covers  administrative  work  in  pro¬ 
viding  clerical  and  general  management  support. 

•  Engineering  process  covers  all  engineering  services.  This  process  is 
divided  into  two  subprocesses  (engineering  and  engineering  techni¬ 
cal  support). 

All  supervisory  positions  were  consolidated  into  one  process.  As  a  re¬ 
sult,  supervision  is  no  longer  equated  with  the  top  grades  in  each  pay 
schedule.  Workers  can  enter  supervision  as  a  distinct  career  field  if 
they  choose  to  do  so,  but  they  do  not  have  to  enter  supervision  to 
progress  to  the  highest  pay  levels  within  their  ovm  process. 

One  intent  of  job  series  consolidation  was  to  reduce  the  time  and 
complexity  involved  in  operating  the  classification  system.  Another, 
broader  intent  was  to  enable  workers  to  be  utilized  on  a  wider  range 
of  tasks,  after  they  had  been  appropriately  trained.  Thus,  workers 
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could  gain  training  and  experience  in  a  wider  variety  of  work  than 
under  the  conventional  classification  system,  which  offered  few  oppor¬ 
tunities  for  training  or  work  outside  narrowly  assigned  classifications. 
Expanding  career  and  multiple-skill  training  opportunities  would 
enrich  the  quality  of  work  life.  Supervisors  would  also  gain  greater 
capability  for  prompt  reassignment  of  workers  in  response  to  changes 
in  the  flow  and  composition  of  work.  By  allowing  a  worker  to  be  as¬ 
signed  to  any  job  in  a  process  for  which  he  is  qualified,  the  system 
was  intended  to  improve  Directorate  responsiveness  to  work  and  mis¬ 
sion  requirements.  Job  series  consolidation  would  use  substantially 
fewer  classifications  to  reflect  differences  in  type  of  work  and  duties. 


Revised  Base  Pay  Determination 

As  a  complement  to  job  series  consolidation,  base  pay  determination 
was  revised  to  incorporate  pay  banding  and  exclude  performance  re¬ 
views.  This  simplified  pay  progression  and  eliminated  peformance 
appraisal  as  a  factor  in  determining  base  pay.  These  revisions  have 
three  m^or  objectives;  (1)  to  support  the  new  classification  system  by 
giving  managers  more  flexibility  in  assigning  work,  (2)  to  give  em¬ 
ployees  a  wider  range  of  potential  salary  growth  without  the  need  for 
formal  promotion  procedures,  and  (3)  to  decrease  reliance  on  individ¬ 
ual  incentives  in  favor  of  an  improved  organizational  consciousness. 

Pay  Banding.  Pay  banding  is  the  combination  of  sets  of  a(^acent 
pay  grades  into  bands,  resulting  in  a  simplified  compensation  system. 
Broadening  pay  categories  was  designed  to  improve  responsiveness  to 
mission  requirements  by  making  a  larger  number  of  potentially  quali¬ 
fied  employees  eligible  for  required  jobs  within  a  category. 

Under  this  intervention,  the  traditional  GS  and  FWS  systems  with 
their  constituent  grades  were  replaced  by  three  pay  schedules  with 
just  four  bands  each  (see  Table  2); 

1.  Demonstration  hourly  (DH),  covering  all  wage  grade  (WG)  and 
wage  leader  (WL)  nonsupervisory  positions. 

2.  Demonstration  salaried  (DW),  covering  all  GS  nonsupervisory 
positions. 

3.  Demonstration  supervisory  (DX),  covering  all  supervisory  positions 
(GS,  GM,  andWS). 

Workers  converted  to  the  new  schedules  at  their  current  earnings 
level.  The  new  system  relies  on  a  guaranteed  annual  percentage  in- 
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Table  2 

Demonstration  Pay  Schedules 
and  Bands 


New  Pay  Schedule 

and  Band  Current  Grades 

Demonstration  Hourly 

DH-1 

WG-ltoWG-3 

DH-2 

WG-4  to  WG-8 

DH-3 

WG-9  to  WG-11 

DH-4 

WG-12  to  WG-15 

Demonstration  Salaried 

DW-1 

GS-1  to  GS-4 

DW-2 

GS-5  to  GS-8 

DW-3 

GS-9  to  GS-12 

DW-4 

GS-13toGS-14 

Demonstration  Supervisory® 

DX-1 

GS-5  to  GS-8 

DX-2 

GS-9  to  GS-12 

DX-3 

GS-13toGS-14 

DX-4 

GS-15 

“This  grouping  of  pay  levels  is  also 
used  for  blue-collar  supervisors  who 
were  converted  based  on  annualizing 
their  current  hourly  rate  of  pay. 


crease  in  salary  within  each  pay  band.  The  system  was  designed  for 
progression  through  each  DH  band  in  12  years;  DW-1  through  DW-3, 
DX-1  and  DX-2  in  25  years  each;  DW-4  and  DX-3  in  16  years  each; 
and  DX-4  in  11  years.  These  times  were  chosen  to  approximate 
movement  under  the  conventional  system,  including  within-grade, 
quality  step,  and  merit  increases,  and  promotions.  With  the  excep¬ 
tion  of  the  DW-4,  DX-3,  and  DX-4  bands,  the  percentage  of  annua]  in¬ 
crease  was  intended  to  be  higher  during  the  first  half  of  the  applica¬ 
ble  period. 

The  demonstration  pay  schedules  are  adjusted  when  a  general 
increase  changes  the  GS.  The  demonstration  hourly  schedule  can 
also  be  adjusted  according  to  Sacramento-area  wage  survey  results. 

As  is  true  for  the  conventional  system,  the  pay  ranges  for  the  bands 
overlap,  so  that  a  new  employee  may  earn  less  than  a  senior  employee 
in  a  lower  band,  In  addition  to  within-band  increases,  employees  may 
earn  salary  increases  by  promotion  to  open  positions  in  a  higher  band. 
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Eliminating  Ir  lividuai  Annual  Performance  Ratings.  Deming 
(19S7)  has  hypothesi7ed  that  individual  performance  appraisal  is 
cou’  :  erproductive  because  it  “nourishes  short-term  performance,  an¬ 
nihilates  long-term  planning,  builds  fear,  diminshes  teamwork,  nour¬ 
ishes  rivalry  and  politic.-;.’’  Under  this  hypothesis,  it  is  not  competi¬ 
tion  among  workers  that  improves  work  quality  but  cooperation,  and 
cooperation  is  especially  important  in  an  organization  such  as  DS 
where  work  units  are  interdependent.  Instead  of  individual  ap¬ 
praisal,  Deming  advocates  more  careful  selection  and  placement  of 
employees,  better  training  and  education,  improved  leadership  and 
counseling,  and  stati.stical  process  control,  which  seeks  to  minimize 
variations  m  work  quality. 

Un  .l.m  PACER  SHARE,  annual  performance  appraisals  with  their 
performance  elements,  st.indards,  and  achievement  ratings  are  no 
longer  used  as  a  basis  lor  movement  within  the  pay  bands.  This  in¬ 
tervention  dispenses  with  individual  ratings,  allowing  the  time  and 
effoi^  entailed  in  producing  them  to  be  allocated  elsewhere. 

Nonetheless,  some  individual  incentives  remain.  Employees  may  still 
be  promoted  from  band  to  band  (without  the  time-in-grade  require¬ 
ments  of  the  old  system),  and  they  should  find  their  career  opportu¬ 
nities  enhanced  through  increased  cross-training.  Moreover,  al¬ 
though  individual,  as  opposed  to  organizational,  incentives  are  limited 
under  PACER  SHARE,  it  is  uncertain  whether  this  represents  a 
meaningful  decrease  in  individual  incentives  from  the  previous  sys¬ 
tem.  Only  a  very  small  percentage  of  the  work  force  received  quality 
step  increases,  and,  even  under  the  Performance  Management  and 
Recognition  System,  employees  may  have  been  rewarded  as  much  in 
rotation  as  on  the  basis  of  outstanding  performance. 

Revised  Supervisory  Grading  Criteria 

In  the  conventional  system,  supervisors’  grades  traditionally  are 
based  in  part  on  the  number  and  grades  of  the  employees  supervised.® 
Under  this  intervention,  supervisors’  grades  are  based  on  factors  that 
reflect  the  supervisors’  job  responsibilities  and  the  difficulty  of 
carrying  out  those  responsibilities.  The  intervention  thus  eliminates 
the  need  for  a  specific  subordinate  structure  for  each  supervisory- 
position,  allowing  supervisors  to  be  assigned  to  positions  where  they 


^As  authorized  by  the  DoD  Appropriations  Act,  the  government  is  implementing 
new  supervisory  grading  criteria  that  do  not  explicitly  rely  on  number  and  gi-ades  of 
employees  supervised.  Whether  such  factors  arc  indeed  removed  from  consideration  is 
yet  to  be  seen. 
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are  most  needed.  Since  salaries  are  not  based  on  the  numbers  and 
grades  of  subordinates  supervised,  .  supervisors  are  freer  to 
recommend  appropriate  staffing  changes.  Specifically,  points  are 
assigned  to  each  supervisory  job  on  six  factors;  workload  of  organiza¬ 
tional  unit;  position  criticality;  degree  and  scope  of  responsibility  del¬ 
egated;  level  and  purpose  of  contacts;  kind,  degree,  and  character  of 
supervision  exercised;  and  planning  horizon.  The  total  number  of 
points  accumulated  across  the  factors  is  used  to  assign  each  supervi¬ 
sory  position  to  one  of  the  four  bands  in  the  DX  schedule.® 

Revised  Hiring/Retention  Criteria 

The  revised  hiring/retention  criteria  are  part  of  a  new  Demonstration 
On-Call  (DOC)  program,  replacing  the  previous  on-call  hiring  author¬ 
ity.  New  employees  are  normally  hired  into  the  DOC  program  and 
are  subject  to  ten  days  notice  for  release  and  three  days  for  recall. 
DOC  employees  are  eligible  for  career  status  after  one  year,  but  con¬ 
version  depends  on  DS  staff  needs,  and  those  with  the  longest  tenure 
are  taken  first.  When  workload  or  budgetary  changes  require  adjust¬ 
ing  the  size  of  the  work  force,  managers  try  to  confine  the  effects  to 
the  applicable  DOC  segment,  accounting  for  veteran  preference  and 
seniority.  Recall  is  in  the  reverse  order.  In  addition,  formal  reduc¬ 
tion-in-force  procedures  are  not  required  for  termination  of  DOC  em¬ 
ployees  if  a  RIF  is  mandated.  This  should  provide  considerable  time 
and  m.^'netary  savings.  However,  the  new  on-call  program  includes  a 
benefits  package,  in  contrast  to  the  previous  one. 


Productivity  Gainsharing 

Productivity  gainsharing  is  a  system  based  on  total  organizational 
performance  in  which  cost  savings  generated  during  the  demonstra¬ 
tion  are  shared  equally  between  the  Air  Force  and  DS  employees. 
Cost  savings  equal  the  difference  between  what  a  workload  would 
have  cost  under  the  existing  system  and  what  it  actually  cost  under 
the  demonstration.  Work  quality  and  timeliness  must  be  maintained 
at  acceptable  levels,  and  the  “would  have”  cost  is  subject  to  periodic 
adjustment  for  such  factors  as  the  introduction  of  cost-saving  technol¬ 
ogy  or  work  methods.  (The  “would  have”  cost  remains  the  same  for  a 
year  if  means  of  “working  smarter”  are  introduced  at  employee 
suggestion  and  for  six  months  if  they  result  from  other  sources.)  Cost 
savings  computations  are  confined  to  operations  and  support  costs 


®For  a  more  complete  description  of  the  six  factors,  sec  Federal  Register,  Vol.  52, 
p.  44792. 
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and  exclude  capital  costs  (plant  and  most  equipment).  Over  90 
percent  of  operations  and  support  costs  are  for  labor;  therefore,  most 
cost  savings  come  from  reduced  labor  costs.  That  is,  cost  savings  un¬ 
der  the  gainsharing  system  are  realized  only  if  the  same  work  is  per¬ 
formed  for  fewer  labor  dollars  or  more  work  is  performed  for  the  same 
labor  cost.  Unless  the  workload  and  funding  for  DS  are  increased,  the 
major  source  of  cost  savings  is  the  ability  of  the  work  force  to  absorb 
the  workload  of  employees  who  leave  through  natural  attrition  pro¬ 
cesses. 

This  system  offers  an  extrinsic  incentive  to  DS  employees  intended  to 
help  them  and  the  Air  Force  take  advantage  of  the  opportunities  for 
greater  productivity  that  should  accompany  the  changes  in  the  per¬ 
sonnel  system.  It  similarly  should  provide  greater  incentive  to  partic¬ 
ipate  in  DS’s  quality  circles,  process  action  teams,  and  task  forces. 
These  programs  contribute  to  a  work-team-based  environment,  pro¬ 
moting  employee  participation  in  identifying  and  solving  organiza¬ 
tional  problems  related  to  work  quality  and  productivity.^® 

The  intention  is  to  link  organizational  performance  directly  to  indi¬ 
vidual  compensation.  Gainsharing  payments  are  linked  to  the  per¬ 
formance  of  the  organization  as  a  whole  rather  than  to  the  perfor¬ 
mance  of  divisions  or  branches.  The  purpose  of  organizational-level 
payments  is  to  compensate  for  inequity  of  opportunity  to  earn  gain- 
shares  in  some  divisions  or  branches  (and  branches  might  not  set 
productivity  criteria  consistent  with  DS-wide  criteria).  Directorate- 
level  gainshares  also  are  more  in  keeping  with  PACER  SHARE’S  cor¬ 
porate  focus.  Payments  to  DS  employees  are  made  in  equal  dollar 
shares  rather  than  being  based  on  a  percentage  of  salary, 

RISKS  OF  THE  DEMONSTRATION 

Despite  the  goals  of  PACER  SHARE  and  the  expected  benefits  of  the 
interventions,  there  may  be  risks  in  the  demonstration.  Organiza¬ 
tional  change  involves  risk  whenever  there  is  uncertainty  about  how 
to  implement  specific  changes  and  the  range  of  their  possible 
outcomes.  If  there  were  no  uncertainty  and  the  expected  outcomes 
were  all  positive,  the  changes  would  be  made  immediately,  barring 
some  overriding  constraint  or  regulation  beyond  the  organization’s 
control.  Analysts  as  well  as  proponents  of  the  demonstration  must  be 
aware  of  downside  risks,  not  only  to  be  sure  that  the  evaluation 
framework  accounts  for  them  along  with  the  expected  benefits,  but 


lOQuality  circles,  begun  in  1980,  predate  PACER  SHARE. 
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also  to  help  formulate  hypotheses  about  why  the  interventions  were, 
or  were  not,  effective.  Downside  risk  means  that  the  organization 
may  become  worse  off  under  the  demonstration  as  a  whole  (or  under 
particular  changes)  than  it  would  have  been  without  it. 

Negative  outcomes  could  arise  for  a  variety  of  reasons,  including  the 
following: 

•  Negative  feedback  created  by  eliminating  performance  appraisals. 
As  pay  increases  will  not  be  tied  to  individual  performance  through 
appraisals,  workers’  incentives  to  shirk  could  increase,  leading  to 
lower  organizational  productivity. 

•  Inefficient  expansion  of  supervisory  positions.  As  fewer  subordi¬ 
nate  positions  will  be  needed  to  justify  a  supervisory  position,  the 
latter  could  be  inefficiently  proliferated. 

•  Higher  outflow  of  workers  due  to  expanded  training.  Under 
PACER  SHARE,  employees  will  be  more  broadly  trained  and  thus 
should  have  more  opportunities  to  work  outside  DS.  If  compensa¬ 
tion  does  not  keep  pace  with  offers  from  alternative  employers, 
workers  could  leave  DS,  causing  the  Directorate  to  bear  the  costs  of 
training  but  not  reap  its  benefits. 

•  More  transfers  out  of  DS  because  of  the  desire  to  earn  promotions 
eliminated  by  pay  banding  or  higher  separation  among  those 
whose  near-term  step  increases  under  the  previous  system  would 
have  exceeded  their  annual  pay  adjustments  under  PACER 
SHARE. 

•  Unexpectedly  fast  wage  growth.  If  the  algorithm  used  to  compute 
annual  pay  growth  results  in  greater  growth  than  would  have  oc¬ 
curred  without  PACER  SHARE,  costs  could  increase  rather  than 
decrease. 

The  evaluation  of  the  demonstration  is  structured  to  account  for  these 
and  other  negative  outcomes  that  might  result  and  to  examine  inter¬ 
relationships  among  the  interventions  that  might  account  for  difficul¬ 
ties  in  achieving  the  project’s  goals  (for  example,  failure  to  provide  the 
training  needed  to  take  advantage  of  the  opportunities  for  organiza¬ 
tional  flexibility  provided  by  series  consolidation  and  pay  banding). 
Although  it  is  necessary  to  plan  the  evaluation  to  account  for  the  pos¬ 
sibility  of  negative  outcomes,  the  demonstration  was  undertaken  with 
the  expectation  that  its  benefits  would  outweigh  any  costs. 

Risk  is  inherent  not  only  in  the  demonstration’s  outcomes  but  in  the 
way  it  is  implemented.  For  example,  a  key  to  effective  startup  lies  in 
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reducing  the  forces  initially  impeding  cooperation  (e.g.,  reducing 
threats)  while  increasing  favorable  forces  (e.g.,  incentives).  This  is 
discussed  more  fully  in  OPM’s  implementation  report. 

EVALUATING  THE  DEMONSTRATION 

RAND’s  evaluation  is  intended  to  measure  the  extent  to  which  the 
goals  and  risks  of  the  PACER  SHARE  demonstration  are  realized. 
Section  2  describes  in  detail  the  criteria  and  means  employed  in  as¬ 
sessing  the  project’s  outcomes,  along  with  the  measures  used. 
Basically,  we  hypothesize  a  set  of  outcomes  in  the  form  of  an  evalua¬ 
tion  model.  We  then  compare  the  outcomes  at  the  Sacramento  Air 
Logistics  Center  to  the  values  of  identical  variables  before  the  demon¬ 
stration  and  to  the  changes  on  these  variables  at  other  Air  Logistics 
Centers  not  subject  to  the  demonstration.  Measures  of  organizational 
flexibility  and  quality  of  work  life  were  developed  by  RAND  (drawing, 
in  many  cases,  from  OPM-defined  variables).  They  are  evaluated 
from  survey  data  collected  by  RAND  (Sec.  3  gives  results  to  date)  and 
personnel  data  routinely  collected  by  the  Air  Force  (see  Sec.  4). 
Measures  of  work  quality  and  timeliness  (Sec.  5)  and  cost  savings 
(Sec.  6)  also  were  developed  by  RAND  and  evaluated  with  data 
gathered  from  Air  Force  records.  Our  conclusions  as  of  the  end  of 
demonstration  year  two  are  given  in  Sec.  7.^^  For  a  demonstration  of 
this  magnitude,  however,  a  substantial  evaluation  period  is  required 
before  firm  conclusions  can  be  drawn  to  guide  future  policy. 


llOffice  of  Personnel  Management,  1989. 

IZpredemonstration  (baseline)  data  are  summarized  for  comparison  in  Secs.  3 
through  6.  More  detailed  baseline  findings  are  documented  in  R-3753-FMP. 


2.  THE  PACER  SHARE  EVALUATION;  METHODS, 
MEASURES,  AND  DATA  SOURCES 


To  measure  the  effects  of  the  five  PACER  SHARE  interventions, 
RAND  has  designed  an  evaluation  that  assesses  the  extent  to  which 
PACER  SHARE  realizes  its  goals.  This  design  has  been  worked  out 
in  collaboration  with  the  Directorates  of  Distribution,  Personnel,  and 
Accounting  and  Finance  at  McClellan  Air  Force  Base  and  with  the 
Office  of  Personnel  Management.  Because  the  demonstration  is  being 
conducted  in  a  natural  environment  rather  than  under  controlled  lab¬ 
oratory  conditions,  it  is  quasi-experimental  in  nature.  The  evaluation 
employs  a  comparison  group  to  help  determine  the  effects  of  the 
demonstration.  Four  other  Air  Logistics  Centers  throughout  the 
country  perform  functions  similar  to  those  of  the  Sacramento  ALC 
and  collectively  serve  as  the  comparison  group;  they  are  not  partici¬ 
pating  in  the  demonstration  but  provide  the  same  tjrpes  of  data  that 
are  collected  at  Sacramento.  The  four  ALCs  are  located  at  Hill  Air 
Force  Base,  Ogden,  Utah;  Kelly  Air  Force  Base,  San  Antonio,  Texas; 
Robins  Air  Force  Base,  Warner-Robins,  Georgia;  and  Tinker  Air  Force 
Base,  Oklahoma  City,  Oklahoma. 

The  evaluation  is  designed  to  identify  the  effects  of  the  interventions 
by  comparing  the  test  site  with  the  comparison  sites  before  and  dur¬ 
ing  the  demonstration  period.  In  any  such  demonstration,  many  fac¬ 
tors  can  change  over  time,  from  the  scope  of  the  organization’s  mis¬ 
sion  to  its  workload,  production  technology,  factor  costs,  incentives, 
and  rewards.  Therefore,  the  demonstration  should  not  be  viewed  as  a 
tightly  controlled  experiment  in  which  an  isolated  factor  is  permitted 
to  vary  while  others  are  not.  By  using  baseline  (predemonstration) 
data  as  well  as  comparison  sites,  the  evaluation  is  structured  to  ask 
(1)  whether  quality  of  work  life,  organizational  flexibility,  quality  and 
timeliness  of  work,  and  productivity  improved  at  the  Sacramento 
ALC  during  the  demonstration,  and  (2)  whether  they  did  so  to  a 
greater  extent  than  expected  without  the  interventions.  A  compari¬ 
son  of  Sacramento’s  performance  during  the  demonstration  with  its 
performance  at  baseline  should  tell  whether  improvement  occurred. 
To  judge  whether  Sacramento’s  performance  improved  relative  to 
what  was  expected,  its  performance  is  compared  with  that  of  the 
other  ALCs  taken  as  a  group.  Their  combined  performance  reflects 
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general,  systemwide  tendencies  and  provides  a  measure  of  how  well 
Sacramento  might  have  done  on  average.^ 

How  comparable  are  the  ALCs?  They  share,  for  example,  many  char¬ 
acteristics,  including  similar  workloads,  job  standards,  and  personnel 
practices.  Nevertheless,  among  the  ALCs  there  are  specific  differ¬ 
ences  in  factors  that  affect  job  performance,  the  most  obvious  example 
being  the  physical  layout  of  the  DSs.  Such  differences  imply  that  the 
times  allotted  for  the  performance  of  specific  tasks  vary  across  the 
ALCs,  even  though  the  same  methods  are  used  in  computing  the 
times.  There  also  may  be  differences  in  the  composition  and  compen¬ 
sation  of  the  work  force  among  the  ALCs  that  could  affect  the  outcome 
measures  being  evaluated. 

By  and  large,  the  similarities  and  differences  among  the  ALCs  tend  to 
persist  through  time,^  making  cross-ALC  comparisons  meaningful 
over  the  course  of  the  demonstration.  Furthermore,  what  change 
does  occur  generally  is  directed  by  Headquarters,  Air  Force  Logistics 
Command,  so  there  is  a  tendency  for  all  ALCs  to  change  in  the  same 
way.  Moreover,  by  agreement  with  the  Air  Force,  no  unique  policy 
changes  will  be  applied  to  Sacramento,  nor  will  Sacramento  be  ex¬ 
empted  from  policy  changes  affecting  other  ALCs. 

The  general  persistence  of  characteristics  and  the  tendency  for  any 
change  to  occur  systemwide  fit  comfortably  into  the  analytic 
paradigm  set  forth  above.  Differences  among  the  ALCs  at  the  outset 
of  the  demonstration  are  netted  out  by  comparing  outcomes  at 
Sacramento  and  the  comparison  group  to  their  own  baselines.  Effects 
of  systemwide  changes  should  be  picked  up  in  trends  at  the  compari¬ 
son  group.  Subtracting  the  latter  from  the  trend  at  the  demonstra¬ 
tion  site  should  then  yield  evidence  of  the  demonstration’s  effects. 

To  evaluate  the  demonstration,  it  is  necessary  to  identify  expected 
outcomes  by  intervention,  define  operational  measures  of  the  out¬ 
comes,  and  specify  the  data  sources.  As  Table  3  indicates,  each  inter¬ 
vention  has  expected,  measurable  effects,  as  does  the  combination  of 
interventions.  Nonetheless,  because  the  demonstration  implements 
all  interventions  at  the  same  time  and  at  a  single  site,  the  effects  of 
any  single  intervention  cannot  be  isolated  from  the  others.  The  ef- 


^In  addition,  the  companion  volume  (N-3404-FMP)  gives  data  for  each  ALC.  These 
data  allow  trends  to  be  detected  and  pairwise  comparisons  to  be  made.  Such  compar¬ 
isons  could  lead  to  a  more  informed  judgment  about  the  combined  ALC  comparisons. 

^Based  on  information  received  from  Sacramento  and  Headquarters,  Air  Force 
Logistics  Command,  as  well  as  on  our  own  data  analyses. 
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fects  of  the  demonstration  must  be  viewed  as  arising  from  the  set  of 
interventions.  These  effects  can  be  grouped  into  four  categories  cor¬ 
responding  to  the  broad  goals  of  the  demonstration,  which  are  to  im¬ 
prove  the  following: 

•  Productivity  (cost  savings) 

•  Organizational  flexibility 

•  Quality  of  work  life 

•  Quality  and  timeliness  of  work.^ 

We  break  the  rest  of  this  section  into  subsections  corresponding  to 
each  of  the  demonstration’s  goals.  (For  the  discussion  that  follows, 
we  group  organizational  flexibility  and  quality  of  work  life  together 
because  the  same  instruments  and  sources  are  used  to  derive  mea¬ 
sures  of  achieving  those  two  goals.)  Within  each  subsection,  we  dis¬ 
cuss  the  data  sources  (summarized  in  Table  4)  and  methodologies 
employed  to  determine  whether  the  demonstration  is  meeting  the 
goal  under  consideration.  We  also  present  results  from  analyf^es  that 
aided  in  the  design  or  validation  of  the  methodological  tools.  Results 
of  the  evaluation  produced  by  the  tools  themselves  are  given  in  Secs. 
3  through  6. 

MEASURING  PRODUCTIVITY:  ANALYSIS  OF  COST 
SAVINGS 

The  cost  analysis  attempts  to  determine  whether  productivity  is  ob¬ 
served  to  increase  under  PACER  SHARE  more  rapidly  than  other¬ 
wise  expected,  where  productivity  increase  is  reflected  through  lower 
output  cost.^  A  fundamental  question  is  how  expected  cost  should  be 
characterized.  The  analysis  examines  expected  cost  from  two  per- 


^We  do  not  explicitly  evaluate  one  of  the  demonstration's  goals — sustaining  or  im¬ 
proving  mobilization  capability  during  emergencies  or  wartime.  We  believe  this  goal 
will  be  achieved  if  organizational  flexibility  and  productivity  improve.  Testing  the  at¬ 
tainment  of  greater  contributions  to  mobilization  would  require  an  exercise,  which 
could  be  conducted  at  a  later  date  if  the  Air  Force  so  desires. 

♦Formally,  we  are  not  studying  productivity;  a  true  analysis  of  productivity  exam¬ 
ines  the  increased  output  for  a  given  bundle  of  inputs  while  holding  the  price  of  those 
inputs  constant.  In  our  case,  cost  savings  might  arise  either  from  increased  productiv¬ 
ity  of  inputs  or  from  decreased  costs  (wages).  As  we  will  see,  however,  Sacramento 
wages  appear  to  be  rising  relative  to  wages  at  the  other  ALCs.  In  the  analysis  below, 
when  we  compare  Sacramento’s  cost  savings  with  cost  savings  at  the  other  ALCs,  the 
fact  that  Sacramento’s  wages  rose  more  rapidly  means  that  any  cost  savings  observed 
will  understate  productivity  gains  there. 
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spectives.  The  first  is  whether  the  rate  of  cost  growth  at  Sacramento 
is  slower  under  PACER  SHARE  than  would  have  been  projected  from 
Sacramento’s  base  period  experience.  The  second  concerns  Sacramen¬ 
to’s  cost  growth  relative  to  that  experienced  by  the  other  ALCs. 
Slower  cost  growth  is  consistent  with  a  positive  effect  of  PACER 
SHARE  on  productivity,  but  it  is  not  strong  evidence  because  other 
changes  can  occur  contemporaneously  with  PACER  SHARE  that  also 
affect  cost  growth.  Such  changes  fall  into  two  categories,  system-level 
changes  such  as  those  promulgated  by  the  Air  Force  Logistics 
Command,  which  has  oversight  for  the  five  ALCs,  and  changes  spe¬ 
cific  to  Sacramento  and  each  other  ALC.  We  control  for  system 
changes  by  analyzing  the  combined  behavior  of  the  other  ALCs.  In 
particular,  using  regression  analysis,  we  compare  cost  growth  at  the 
other  ALCs  before  and  during  PACER  SHARE  with  that  occurring  at 
Sacramento,  to  evaluate  Sacramento’s  performance  relative  to  that 
expected  without  the  demonstration.  Regressions  cannot  be  used, 
however,  to  isolate  ALC-specific  changes  umelated  to  the  demonstra¬ 
tion.  If  such  changes  occur  at  Sacramento,  affect  cost,  and  are  con¬ 
temporaneous  with  PACER  SHARE,  they  will  be  intertwined  with  the 
effects  of  PACER  SHARE  in  the  cost  estimates  for  Sacramento. 
Similarly,  such  changes  at  other  ALCs  may  affect  their  cost  estimates. 

Variables  Measured.  The  cost  analysis  focuses  on  labor  cost,  which 
constitutes  over  90  percent  of  short-run  variable  cost  (the  remaining 
costs  stem  primarily  from  shop  supplies  such  as  wood,  staples,  pack¬ 
ing  material,  etc.).  There  are  no  available  data  on  energy  and  capital 
costs,  but  those  costs  should  vary  little  over  the  analysis  period. 
Energy  use  (heating,  lighting)  is  expected  to  remain  approximately 
constant  at  preperiod  levels  during  the  course  of  the  demonstration 
period.  Capital  costs  related  to  plant  and  facilities  also  are  expected 
to  change  little  because  the  same  buildings  and  layout  are  expected  to 
be  maintained  at  the  ALCs.  A  m^jor  change  in  equipment — the 
Automated  Warehouse  System  (AWS) — ^was  introduced  at  all  the 
ALCs  in  the  years  just  before  PACER  SHARE  began.  The  utilization 
level  is  expected  to  be  the  same  across  the  ALCs  once  AWS  is  com¬ 
pletely  phased  in.  The  phase-in  period  will  differ  somewhat  across 
the  ALCs,  but  that  should  have  little  effect  on  our  cost  savings 
analysis.  Output  is  measured  by  monthly  transactions — issues  and 
receipts  of  materiel — between  DS  and  off-base  customers  or  other 
directorates  on  base.  We  obtain  data  on  receipts  from  off  base,  on- 
base  receipts  from  maintenance,  on-base  receipts  not  from 
maintenance,  issues  to  off  base,  on-base  issues  to  maintenance,  on- 
base  issues  not  to  maintenance,  and  on-base  issues  to  disposal. 
Issues  and  receipts  are  associated  with  various  sets  of  tasks  that  in 
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aggregate  amount  to  the  workload.  For  instance,  a  received  package 
is  typically  unloaded,  unpacked,  and  inspected;  its  contents  might 
then  be  placed  in  storage  on  site,  issued  to  maintenemre  for  repair,  or 
perhaps  combined  with  other  materiel  to  be  shipped  to  another 
destination.  Issues  may  require  retrieval  from  storage,  packaging, 
packing,  inspection,  and  transportation  to  the  point  of  shipment. 
Of  course,  there  also  are  a  host  of  related  support  activities  such 
as  quality  and  timeliness  assessments;  inventory  control; 
rewarehousing;  maintaining  a  paper  trail  for  each  item  received, 
stored,  or  issued;  audits;  and  staff  training.  In  addition,  employees 
participate  in  activities  associated  with  more  general  initiatives  such 
as  team  building  and  quality  circles. 

We  concentrate  on  issues  and  receipts  because  they  reflect  the  mis¬ 
sion  of  the  organization.  Since  DS’s  business  is  transshipment,  issues 
and  receipts  are  a  valid  measure  of  output.  An  alternative  measure  of 
output  would  incorporate  internal  support  tasks.  Those  are  used  in 
determining  manning  authorizations  but  are  rarely  used  in  measur¬ 
ing  productivity,  because  they  are  not  necessarily  tied  to  external 
demands  for  services  and  are  thus  subject  to  manipulation.  For  in¬ 
stance,  an  ALC  could  possibly  exaggerate  its  apparent  workload  by 
increasing  the  volume  of  internal  tasks  above  the  minimum  necessary 
to  handle  its  issues  and  receipts.  If  Sacramento  did  so  during  PACER 
SHARE,  for  example,  it  could  give  the  appearance  of  producing  more 
output  with  the  same  paid  hours  of  work,  thereby  making  PACER 
SHARE  seem  effective.  Similarly,  other  ALCs  could  conceivably  do 
the  same  thing,  which  might  make  them  appear  more  productive  rel¬ 
ative  to  Sacramento.  In  actuality,  the  multiplication  of  needless  sup¬ 
port  tasks  is  indicative  of  lower  productivity. 

To  ensure  that  cost  savings  indicate  true  productivity  gains,  we  must 
check  to  see  that  quality  and  timeliness  do  not  decline  under  PACER 
SHARE.  To  accomplish  that,  we  initially  considered  including  quality 
and  timeliness  variables  in  the  regressions.  But  that  is  not  a  viable 
approach  because  of  the  way  quality  and  timeliness  outcomes  are 
generated.  When  a  quality  measure  declines,  for  instance,  steps  are 
soon  taken  to  identify  the  cause  and  correct  it.  As  a  result,  the  ob¬ 
served  mean  and  variance  of  quality  and  timeliness  measures  tend  to 
follow  the  standard  sought  and  its  level  of  tolerance.  Consequently, 
instead  of  the  regression  approach,  we  will  separately  monitor  the 
quality  and  timeliness  indicators.  Quality  and  timeliness  indicators 
are  described  later. 

Data:  Sources  and  Adjustments.  We  use  monthly  labor  cost  and 
output  data  from  routinely  maintained  data  systems  that  are  compa- 
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rable  across  all  ALCs.  The  labor  cost  data — total  payroll  by  month — 
comes  from  their  workload  system  (designated  D012).  (The  base  pe¬ 
riod  includes  monthly  observations  from  October  1984  through 
December  1987,  and  the  “year-two”  data  actually  cover  30  months  of 
the  demonstration  period,  January  1988  through  June  1990.)  These 
data  have  been  adjusted  for  wage  inflation  and  are  stated  in  constant 
1989  dollars,  thus  avoiding  confounding  the  effects  of  inflation  with 
other  effects.  The  cost  inflator  is  based  on  the  figures  shown  in  Table 
5,  taken  from  Chap.  5  of  Air  Force  Regulation  173-13.  The  annual  in¬ 
flation  rate  was  assumed  to  be  valid  at  the  midpoint  of  the  fiscal  year, 
and  the  inflation  rates  for  intervening  months  were  found  by  interpo¬ 
lation  and  for  end  months  by  extrapolation.  For  example,  the  infla¬ 
tion  rate  for  February  1989  is  equal  to  the  inflation  rate  for  March 
1989  (1.000)  plus  one-twelfth  the  difference  of  the  1989  rate  from  that 
of  1988.  The  wage  inflators  shown  in  Table  5  were  further  adjusted 
for  wage  growth  due  to  the  introduction  of  the  Federal  Employee 
Retirement  System,  which  led  to  rapid  growth  in  the  wage  bill  during 
1986  and  1987  as  the  system  was  phased  in. 

Two  sources  of  output  data  are  available.  One  source  is  the  workload 
system  and  the  other  is  the  financial  system  (the  latter  consists  of  the 
system  designated  D033  and  the  Standard  Base  Supply  System).  The 
workload  system  ties  each  issue  or  receipt  to  a  specified  series  of 
transactions  related  to  its  processing,  which  in  turn  are  linked  to  its 
labor  standards;  this  system  is  used  in  scheduling  and  manpower 
planning.  The  workload  system  draws  its  input  from  relevant  finan¬ 
cial-system  records  and  from  manual  entry.  The  workload  output 
data  have  two  disadvantages:  they  count  transactions  rather  than 
counting  issues  and  receipts  directly,  and  they  are  not  auditable.  The 
financial  system  data  avoid  these  disadvantages;  issues  and  receipts 
are  counted  directly  and  are  audited.  Moreover,  a  complete  file  of 
output  data  is  now  available  for  this  system  but  not  the  workload  sys¬ 
tem.  For  these  reasons,  we  use  the  financial  system  data  on  output. 

Table  6 

Wage  Cost  Inflator 


Fiscal 

Year  Inflator 


1985 

1.202 

1986 

1.190 

1987 

1.129 

1988 

1.035 

1989 

1.000 

1990 

.966 
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Econometric  Model 

We  bemn  by  discussing  a  simple  rcOTession  specification  where  (the 
natural  logarithm  (^n)  of  labor  cost  depends  on  time  and  (fn)  output 
level: 

^nct  »  a^  +  a^t +  a2^nxt +0^ 

The  specification  assumes;  (1)  the  labor  cost  change  from  one  month 
to  the  next  occurs  at  a  constant  rate  of  ai,  that  is,  aj  is  the  time  trend 
in  labor  cost,  and  (2)  labor  cost  is  proportionately  related  to  output, 
that  is,  labor  cost  changes  by  a2  percent  for  a  one  percent  increase  in 
output.  The  time  trend  a^  can  be  positive  or  negative  depending  on 
whether  wages  are  rising  faster  than  productivity.  If  productivity 
were  improving  and  wages  were  declining,  a^  would  be  negative.  We 
expect  a-i  to  be  positive  but  less  than  one.  This  is  because  the  demand 
for  labor  should  increase  as  workload  increases,  and  so  the  amount 
spent  on  labor  should  also  increase.  However,  it  is  typically  less 
costly  to  vary  the  work  force  utilization  rate  (proportion  of  time  ac¬ 
tively  engaged  in  work)  and  effort  rate  (output  per  unit  time  when  ac¬ 
tively  working)  than  to  add  or  release  workers.  As  a  result,  part  of 
the  added  work  needed  to  handle  a  bigger  workload  comes  from 
greater  exertion  by  workers  already  on  the  payroll.  Thus,  labor  cost 
will  rise  by  a  smaller  percentage  than  output.® 

For  our  empirical  work  we  basically  expand  the  above  specification  to 
allow  separate  coefficients  for  other  ALCs  during  the  base  period, 
other  ALCs  during  the  demonstration  period,  Sacramento  during  the 
base  period,  and  Sacramento  during  the  demonstration  period.  This 
kind  of  flexibility  is  essential  to  testing  our  cost  hypotheses  (it  is  also 
sensible  in  view  of  changes  that  might  occur  under  PACER  SHARE). 
First,  PACER  SHARE  could  increase  the  rate  of  productivity  im¬ 
provement,  implying  a  lower  time  trend  coefficient  (ai)  for 
Sacramento  during  the  demonstration  than  in  the  base  period. 
Second,  the  change  in  supervisory  grading  criteria  as  well  as  the  use 
of  Demonstration  On-Call  employees  could  allow  Sacramento  to  main¬ 
tain  a  smaller  work  force,  which  would  be  reflected  in  a  lower  inter¬ 
cept  (ao)  in  the  demonstration  period.  Third,  job  series  consolidation, 
revised  base  pay  determination,  and  increasingly  pervasive  training 


5Two  other  variables  normally  would  appear  in  this  model:  the  price  (or  amount)  of 
capital  and  the  real  wage  rate  of  labor  (i.e.,  a<ijusted  for  inflation).  We  do  not  have  data 
on  these  variables  apart  from  an  overall  wage  inllator,  which  we  used  to  ac^just  labor 
cost  but  which  is  not  ALC-specific.  However,  their  absence  will  have  little  effect  on 
empirical  results  because  in  the  short  term  they  change  little. 
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in  multiple  skills  (as  discussed  in  the  previous  section)  could  increase 
the  work  force  capability  to  respond  to  workload  fluctuations.  This 
would  tend  to  make  the  workload  coefficient  (02)  smaller.  That  is,  if  a 
given  work  force  can  handle  a  wider  variety  of  tasks,  and  managers 
have  the  authority  to  assign  workers  promptly  to  those  task?  then 
there  should  be  less  variation  in  work  force  size,  hence  labor  cost,  as 
output  varies.  Possibly  counteracting  this  effect,  Sacramento  could 
make  greater  use  of  its  DOCs,  adding  and  subtracting  them  as 
needed,  which  would  make  labor  cost  more  responsive  to  workload. 
Thus,  the  workload  effect  could  become  larger  or  smaller  than  at 
baseline. 


Previous  Models 

In  the  year-one  cost  analysis  we  specified  and  estimated  two  cost 
models.  One  related  unit  labor  cost  (labor  cost  divided  by  output)  to 
an  intercept  and  time  trend  only.  The  other  model  appeared  similar 
to  the  total  labor  cost  equation  (Eq.  (1)) — cost  is  a  function  of  time  and 
output-,  however,  it  employed  a  “spline”  specification  for  the  time  vari¬ 
able.  Under  the  spline,  the  time  trend  line  in  the  demonstration  period 
is  forced  to  begin  at  the  end  point  of  the  baseline  time  trend  line.  The 
time  spline  allowed  us  to  drop  the  demonstration  period  intercept  and 
estimate  fewer  coefficients  with  our  limited  data. 

The  greater  quantity  of  data  available  for  the  present  analysis  en¬ 
abled  us  to  test  the  adequacy  of  the  previous  models.  These  tests 
found  the  total  labor  cost  model — including  terms  for  both  time  and 
output — superior  to  the  unit  labor  cost  model  (time  only).  We  discov¬ 
ered  that  the  unit  labor  cost  model  should  include  an  output  variable, 
which  also  should  interact  with  the  time  trend.  In  other  words,  unit 
cost  is  not  independent  of  output,  as  the  earlier  model  assumed. 
When  the  unit  cost  model  is  expanded  to  include  output  and  a  time 
trend-output  interaction  it  performs  much  better,  but  still  not  as  well 
as  the  total  labor  cost  model  described  above.  For  this  reason  we 
have  dropped  the  unit  cost  model  from  our  analysis. 

The  previous  total  labor  cost  model  fared  better  in  the  specification 
tests.  The  tests  pointed  to  the  desirability  of  relaxing  the  time  trend 
constraint  by  eliminating  the  spline — as  we  have  now  done  in  the  to¬ 
tal  cost  model  described  above — ^yet  the  improvement  in  fit  from  this 
change  is  minor. 

The  review  of  previous  specifications  gave  no  reason  to  modify  the 
year-one  cost  analysis  conclusion  that  PACER  SHARE  had  not  then 
achieved  statistically  significant  cost  savings. 
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MEASURING  ORGANIZATIONAL  FLEXIBILITY  AND  THE 
QUALITY  OF  WORK  LIFE 

Organizational  flexibility  and  quality  of  work  life  are  both  measured 
using  survey  data  and  data  from  computerized  flies  of  the  Directorate 
of  Personnel.  In  concert  with  the  Office  of  Personnel  Management, 
RAND  developed  an  extensive  survey  of  attitudes  in  the  workplace. 
The  survey  covers  a  broad  range  of  quality  of  work  life  issues,  such  as 
job  satisfaction,  pay  satisfaction,  organizational  involvement,  motiva¬ 
tion,  and  supervision.  There  also  are  questions  concerning  organiza¬ 
tional  flexibility.  For  example,  supervisors  are  asked  about  their 
perceptions  of  the  job  classification  process,  freedom  to  make  assign¬ 
ments  to  meet  the  workload,  ability  to  meet  changes  in  workload,  and 
criteria  used  for  establishing  supervisory  grades.  The  survey  includes 
numerous  0PM  core  items  previously  tested  for  their  reliability  and 
validity  that  are  meant  to  provide  comparisons  with  other  demonstra¬ 
tions  as  well  as  a  variety  of  questions  specifically  tailored  for  PACER 
SHARE.  Most  of  the  items  consist  of  brief  statements  followed  by  a 
five-point  Likert  scale  indicating  level  of  agreement  with  the  items.® 
The  survey  was  administered  before  project  implementation  and  is 
being  administered  annually  to  DS  employees  at  the  Sacramento  ALC 
and  the  comparison  sites. 

Changes  in  the  quality  of  work  life  also  are  measured  with  nonsurvey 
data  derived  from  information  contained  in  the  records  and  computer¬ 
ized  databases  of  the  Directorate  of  Personnel.  The  measures  include 
separations  and  migrations  (internal  transfers),  among  others.  These 
data  were  collected  for  1987  to  establish  a  baseline  and  are  being  col¬ 
lected  annually  to  assess  the  effects  of  PACER  SHARE. 

Organizational  flexibility  also  is  assessed  using  personnel  measures. 
These  measures  in  the  automated  personnel  database  system  include 
factors  such  as  the  skill  base  of  the  work  force,  supervisory  levels,  and 
pay-related  measures.  In  future  years,  some  measures  of  flexibility 
are  to  be  measured  through  a  special  survey  analysis  developed  by 
0PM,  namely,  the  Personnel  Office  Productivity  Analysis.  This  is  a 
survey  of  classification  actions,  number  of  applications,  and  other 
personnel  office  worker  output  (measured  by  quarter)  and  the  time 
required  to  generate  such  outputs  (measured  over  a  two-week  period 
each  quarter).  Special  audits  of  personnel  records  are  to  be  used  for 
other  flexibility  measures  (classification  error  rate  and  the  cost  of  any 
reduction  in  force  that  might  be  implemented). 


®See  App.  A  in  N-3404-FMP  for  the  full  questionnaire. 
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The  analytical  framework  for  assessing  changes  in  measures  of  orga¬ 
nizational  flexibility  and  quality  of  work  life  provides  the  means  to 
determine  the  statistical  significance  of  three  types  of  effects: 

•  Differences  in  organizational  flexibility  and  quality  of  work  life  be¬ 
tween  the  Sacramento  ALC  and  the  comparison  site  group  prior  to 
the  demonstration,  that  is,  the  measure’s  baseline  value  at 
Sacramento  minus  its  value  for  the  comparison  site  group. 

•  Changes  in  organizational  flexibility  and  quality  of  work  life  within 
the  Sacramento  ALC  and  the  comparison  group  during  the  demon¬ 
stration  period. 

•  Differences  between  the  extent  of  change  at  Sacramento  and  at  the 
comparison  site  group  relative  to  the  baseline  values  for  each,  that 
is,  change  in  the  measure’s  value  at  Sacramento  minus  change  in 
its  value  for  the  comparison  group. 

For  nonsurvey  measures,  both  multivariate  regression  analyses  and 
tests  of  the  difference  in  proportions  (or  means)  between  the 
Sacramento  ALC  and  the  comparison  ALCs  are  conducted.  Survey 
measures  are  analyzed  using  multivariate  regression.  Many  of  the 
individual  items  in  the  survey  are  combined  into  more  broadly  de¬ 
fined  scales,  based  both  on  previous  analyses  of  similar  items  and  the 
results  of  factor  analyses.  The  broad  scales  cover  such  areas  as  in¬ 
trinsic  work  satisfaction,  organizational  climate,  and  adequacy  of  su¬ 
pervision. 


Survey  Methods  and  Measures 

Survey  Administration.  The  baseline  survey  was  administered  to 
personnel  of  the  Directorate  of  Distribution  at  each  of  the  Air 
Logistics  Centers  during  the  period  from  late  fall  1987  through  winter 
1988,  just  before  the  demonstration  started.  (See  R-3753-FMP  for 
administration  and  response  rate  detail.)  The  second-year  survey 
was  administered  in  March  1990  (see  Table  6).  The  survey  sample 
consisted  of  all  DS  employees  at  the  Sacramento  ALC  and  550  ran¬ 
domly  chosen  nonsupervisors  plus  all  DS  supervisory  personnel  at 
each  of  the  four  other  ALCs  (about  one-third  of  the  work  force). 
Supervisors  were  oversampled  to  strengthen  separate  analyses  of 
their  results,  given  that  they  compose  only  10  percent  of  the  work 
force.  Overall,  we  chose  sample  sizes  based  on  a  desire  to  detect  atti¬ 
tude  changes  of  one  point  on  a  five-point  scale  among  10  percent  of 
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Table  6 

Second-Year  Survey  Administration  Dates  and  Response  Rates 


ALC 

Dates 

Administered 

Number  of 
Respondents 

Sample 

Size 

Gross  Response 
Rate,  % 

Sacramento 

20-23  Mar  90 

1196 

1587 

75.4 

Ogden 

27  Mar  90 

593 

758 

78.2 

Oklahoma  City 

15  Mar  90 

603 

783 

77.0 

San  Antonio 

13-14  Mar  90 

598 

819 

73.0 

Wamer-Robins 

29-30  Mar  90 

399 

772 

51.7 

Total/average 

3389 

4719 

71.8 

the  work  force  with  a  reasonable  degree  of  statistical  precision 

(p<.10). 

The  baseline  questionnaire  consisted  of  150  multiple  choice  questions 
concerning  respondents’  attitudes  toward  their  work  environment,  22 
background  and  demographic  questions,  and  four  items  for 
Sacramento  employees  concerning  PACER  SHARE.  The  questions 
were  drawn  from  a  standard  list  prepared  by  the  0PM  for  use  and 
comparison  in  demonstration  projects  supplemented  by  specific  items 
developed  by  RAND  (in  consultation  with  DS)  for  PACER  SHARE. 
Seventeen  questions  were  added  to  the  survey  for  the  first-year  and 
subsequent  follow-ups.  Most  of  those  dealt  with  team  building  and 
other  aspects  of  workers’  sense  of  participation  in  DS’s  mission. 

RAND  staff  administered  the  survey  on  site  in  group  sessions,  which 
typically  consisted  of  60  to  150  personnel.  To  facilitate  candor,  re¬ 
spondents  were  not  identified  on  the  questionnaires.  (Thus,  it  will 
not  be  possible  to  track  individual  respondents  through  the  course  of 
the  demonstration.)  The  RAND  staff  member  began  with  a  short  ex¬ 
planation  of  the  purpose  of  the  survey  and  answered  any  questions. 
Respondents  then  completed  the  self-administered  form,  working  in¬ 
dependently.  The  average  time  required  to  fill  out  the  second-year 
questionnaire  was  30  minutes. 

Overall,  the  gross  response  rate  across  the  five  ALCs  was  72  percent 
(see  Table  6),  which  represented  a  response  rate  of  75  percent  at 
Sacramento  and  70  percent  for  the  comparison  group  (i.e.,  the  other 
ALCs,  collectively).’^  Supervisory  and  nonsupervisory  response  rates 
were  78  and  74  percent,  respectively,  at  Sacramento  and  73  percent 


^Overall  response  rate  at  baseline  was  also  72  percent;  the  Sacramento  rate  was  80 
percent  and  the  comparison  group’s,  68  percent. 
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and  69  percent  for  the  comparison  group.  An  examination  of  person¬ 
nel  records  for  the  survey  dates  suggests  that  up  to  half  of  the  indi¬ 
viduals  who  did  not  report  for  the  survey  were  on  annual  or  sick  leave 
or  were  otherwise  not  physically  available  to  report  for  the  survey. 
The  refusal  rate  among  those  who  did  report  was  very  low,  amounting 
to  less  than  one  percent  at  each  site. 

Scale  Construction.  Planned  and  exploratory  analyses  were  per¬ 
formed  on  the  baseline  survey  data.  To  begin  with,  the  items  consti¬ 
tuting  each  scale  identified  by  0PM  before  the  baseline  survey  were 
grouped  together  (the  order  of  item  presentation  in  the  questionnaire 
had  been  randomized  to  minimize  the  development  of  response  sets) 
and  Cronbac.h’s  Alpha  Coefficient  was  computed.  The  coefficient  indi¬ 
cates  the  extent  to  which  the  answers  to  the  items  were  correlated 
with  each  other  (i.e.,  the  extent  to  which  the  items  behaved  as  a  scale 
in  the  survey).  Next,  based  on  an  extensive  analysis  of  the  intercorre¬ 
lations  among  the  survey  items  and  the  groupings  of  items  resulting 
from  factor  analyses,  we  modified  several  of  the  0PM  scales  by 
adding  one  or  two  items  whose  responses  were  found  to  be  highly  re¬ 
lated  to  those  of  the  items  originally  included  in  the  scale.  (Results  of 
the  factor  analyses  are  given  in  App.  C  of  N-3146-FMP.)  Item  dis¬ 
criminant  validity  analyses  also  were  performed  to  see  whether  items 
correle+^^ed  more  highly  with  their  own  scales  than  with  other  scales. 
The  resulting  measures  formed  the  final  set  of  “0PM  scales”  used  in 
the  analysis. 

Table  7  shows  the  final  “0PM  scales”  used  in  the  evaluation  and  the 
variables  (i.e.,  survey  questions,  by  number)  making  up  each  scale. 
Modified  scales  are  indicated  by  an  alphabetic  suffix  appended  to  the 
scale  number  (e.g.,  OPM03B);  the  added  item(s)  are  italicized.  The 
table  also  shows  the  Alpha  coefficient  obtained  for  the  scale,  based  on 
the  answers  provided  by  all  of  the  respondents  across  the  five  ALCs 
at  baseline.  The  bracketed  Alpha  coefficient  for  the  modified  scales 
reflects  the  Alpha  obtained  before  the  inclusion  of  the  added  item(s). 

In  some  instances,  a  question’s  polarity  was  reversed  when  it  was 
combined  with  the  other  questions  defining  a  scale  to  generate  the 
mean  scale  score.  For  example,  under  External  Equity  (OPM04), 
strong  agreement  with  V070  indicates  equity,  whereas  strong  agree¬ 
ment  with  V022  indicates  inequity.  Therefore,  in  averaging  each 
individual’s  responses  to  the  two  questions  to  come  up  with  a  scale 
average,  the  Likert  categories  for  V022  were  reversed  in  value.  As  a 
result,  “strongly  disagree”  with  V022  (inequity)  was  given  the  same 
value  as  “strongly  agree”  with  V070  (equity).  In  Table  7,  instances  of 
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Table  7 

OPM  Scales:  Questions  and  Alpha  Coefficients 

Alpha  [Old]  Scale  Code _ Scale  Name  and  Queation _ 

.631  OPM02  CONTROL  OVER  WORK 

VO  17  I  hsve  control  over  how  I  spend  my  time  working. 

VOSS  I  have  a  great  deal  of  say  over  what  has  to  be  done  on  my 
job. 

V071  In  DS,  authority  is  clearly  delegated. 

V077  I  have  the  authority  I  need  to  accomplish  my  work  objec¬ 
tives. 

.646  OPM03B  EXPECTANCY 

[.S19] 

V043  Working  hard  on  my  job  leads  to  good  job  performance. 

V061  Working  hard  on  my  job  leads  to  gaining  respect  from  my  co- 

workers. 

V02S  Coming  up  with  new  ways  to  do  my  job  leads  to  good  job  per¬ 
formance. 

.700  OPM04  EXTERNAL  EQUITY 

••  V022  reversed 

V022  Other  employers  in  this  area  pay  more  than  the  government 
does  for  the  kind  of  work  I  am  doing. 

V070  My  pay  is  fair  considering  what  other  places  in  this  area  pay 
for  the  same  kind  of  work. 

.7S7  OPMOSB  EXTRINSIC  REWARD  IMPORTANCE 

[.669]  (V124-V126: 

How  important  is  each  of  the  following  to  you?) 

V126  Your  chances  for  getting  a  promotion? 

V126  The  amount  of  job  security  you  have? 

V124  Retirement  benefits? 

.8S0  OPM06  GENERAL  SUPERVlSIONyDIRECTION 

V014  My  job  duties  are  clearly  defined  by  my  supervisor. 

VO  19  My  supervisor  handles  the  administrative  parts  of  his/her  job 
well. 

V060  My  supervisor  sets  clear  goals  for  me  in  my  present  job. 

V074  My  supervisor  encourages  me  to  help  in  developing  work 
methods  and  job  procedures. 

V075  My  supervisor  helps  me  solve  work-related  problems. 

V089  On  my  job  I  know  exactly  what  is  expected  of  me. 

.803  OPM07  GROUP  FUNCTIONING 

V004  My  unit  works  well  together. 

VOll  I  have  confidence  and  trust  in  my  co-workers. 

VO  18  My  co-workers  encourage  each  other  to  give  their  best  effort. 
VlOO  All  in  all,  I  am  satisfied  with  my  work  unit. 

.664  OPM08  INTENT  TO  TURN  OVER 

VOSS  I  often  think  about  quitting. 

V105  During  the  next  year  I  will  probably  look  for  a  new  job  out¬ 
side  DS. 
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Table  7 — continued 

Alpha  [Old]  Scale  Code _ Scale  Name  and  Queation _ 

.477  OPM09  INTERNAL  EQUITY 

V048  Pay  differences  in  DS  fairly  represent  real  differences  in  lev¬ 
els  of  responsibility  and  job  difficulty. 

V087  My  pay  is  fair  considering  what  people  in  other  directorates 
are  paid. 

.877  0PM 10  INTRINSIC  REWARD  IMPORTANCE 

(V120-V123: 

How  important  is  each  of  the  following  to  you?) 

V120  Challenging  work  responsibilities? 

V121  The  chance  to  accomplish  something  worthwhile? 

V122  The  chance  to  learn  new  things  on  your  job? 

V123  Getting  a  feeling  of  accomplishment  from  your  job? 

.789  OPMll  INTRINSIC  WORK  SATISFACTION 

V033  My  job  allows  me  to  achieve  personal  satisfaction. 

V086  My  job  is  challenging. 

V090  The  work  I  do  on  my  job  is  meaningful  to  me. 

.837  0PM12  JOB  SATISFACTION 

V054  In  general,  I  like  working  here. 

V058  In  general,  I  am  satisfied  with  my  job. 

.425  OPM13  LOCKING  IN 

••  V030  reversed 

V030  I  could  find  a  job  with  another  employer  with  about  the  same 
pay  and  benefits  as  I  now  have. 

V047  It  would  be  very  hard  for  me  to  leave  my  job  even  if  I  wanted 
to. 

V083  I  have  too  much  at  stake  in  my  job  to  change  jobs  now. 

.674  OPM14  OPEN  GROUP  PROCESS 

*•  V057  reversed 

V005  If  we  have  a  decision  to  make,  everyone  is  involved  in  mak¬ 
ing  it. 

V057  My  co-workers  are  afraid  to  express  their  real  views. 

V068  In  my  work  unit  we  tell  each  other  the  way  we  are  feeling. 
V094  In  my  work  unit  everyone’s  opinion  gets  listened  to. 

.739  OPM15  ORGANIZATIONAL  CLIMATE 

••  V013,  V038.  V052  reversed 

Vo  13  Employees  here  feel  you  can’t  trust  management  in  this  di¬ 

rectorate. 

V038  In  DS,  conflict  that  exists  between  work  units  gets  in  the 
way  of  getting  the  job  done. 

V046  Coordination  among  work  units  is  good  in  DS. 

V052  People  in  DS  will  do  things  behind  your  back. 

V071  In  DS,  authority  is  clearly  delegated. 

V088  Management  is  flexible  enough  to  make  changes  when  neces¬ 
sary. 
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Table  7 — continued 

Alpha  [Old]  Scale  Code  Scale  Name  and  Question 

.583  OPM16  ORGANIZATIONAL  INFLUENCE 

**  V007,  VlOl  reversed 

V007  When  changes  are  made  in  DS,  the  employees  usually  lose 
out  in  the  end. 

VlOl  Employees  do  not  have  much  opportunity  to  influence  what 
goes  on  in  DS. 

.603  0PM 1 7  PAY  AS  A  MOTIVATOR 

V060  The  amount  of  money  I  will  receive  for  working  harder  is 
enough  to  make  me  work  harder. 

V079  I  will  receive  more  money  if  I  work  harder  for  DS. 

.830  OPM18D  PAY-PERFORMANCE  LINK/PERFORMANCE  REWARDS 

[.793]  *•  V084  reversed 

V002  Regular  pay  increases  here  depend  on  how  well  a  person 
performs  his/her  job. 

V024  Promotions  here  depend  on  how  well  a  person  performs 
his/her  job. 

V027  I  will  be  promoted  or  given  a  better  job  if  I  perform  especially 
well. 

V082  I  will  get  a  larger  pay  increase  if  I  perform  especially  well. 

V084  Under  the  present  system  financial  rewards  are  seldom  re¬ 
lated  to  employee  performance. 

V099  I  will  have  better  job  security  if  I  perform  especially  well. 

V106  My  own  hard  work  will  lead  to  recognition  as  a  good  per¬ 
former. 

V029  My  pay  is  determined  by  my  individual  job  performance. 

V059  In  DS,  employees  receive  equal  pay  for  equal  work. 

.805  OPM19  PAY  SATISFACTION 

V008  Considering  my  skills  and  the  effort  I  put  into  my  work,  I  am 
satisfied  with  my  pay. 

V081  All  in  all,  I  am  satisfied  with  my  pay. 

.672  OPM21B  RECONSIDERATION/REDRESS 

[.589] 

V066  In  general,  disciplinary  actions  taken  in  DS  are  fair  and  jus¬ 
tified. 

V076  If  I  were  subject  to  an  involuntary  personnel  action,  I  believe 

I  would  be  told  about  my  grievance  and  appeal  rights. 

V067  Employees  here  take  full  advantage  of  their  grievance  and 

appeal  rights. 

.767  OPM23  SATISFACTION  WITH  PROMOTIONS 

V024  Promotions  here  depend  on  how  well  a  person  performs 
his/her  job. 

V027  I  will  be  promoted  or  given  a  better  job  if  I  perform. 

.531  OPM25  (SUBSCAL.E-.  OTHER  WORK  GROUPS] 

•*  V038  reversed 

V038  In  DS,  conflict  that  exists  between  work  units  gets  in  the 
way  of  getting  the  job  done. 

V046  Coordination  among  work  units  is  good  in  DS. 
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Table  7 — continued 


Alpha  [Old] 

Scale  Code 

Scale  Name  and  Question 

.659 

OPM26 

[SUBSCALE:  PERSONNEL  OFFICE  HELPFULNESS] 

*•  V139,  V149  reversed 

V137 

The  personnel  ofUce  helps  me  perform  my  job  effectively. 

V139 

It  takes  too  long  to  process  the  paperwork  needed  to  fill  va¬ 
cancies  here. 

VU3 

The  personnel  department  here  provides  line  management 
with  valuable  support  services. 

V149 

Supervisors  in  DS  feel  their  ability  to  manage  is  restricted  by 
unnecessary  personnel  rules  and  regulations. 

.681 

OPM27 

SUPERVISORS:  AUTHORITY 

V132 

I  have  enough  authority  to  hire  competent  people  when  I 
need  them. 

V141 

I  have  enough  authority  to  determine  my  employees’  pay. 

V146 

I  have  enough  authority  to  promote  people. 

V150 

I  have  enough  authority  to  remove  people  from  their  jobs  if 
they  perform  poorly. 

.799 

OPM30 

SUPERVISORS:  SATISFACTION 
••  V128.  V139,  V149  reversed 

V128 

It  takes  too  long  to  get  decisions  made  in  DS. 

V133 

Top  management  generally  supports  the  personnel  decisions 
made  by  supervisors  in  DS. 

V137 

The  personnel  office  helps  me  perform  my  job  effectively. 

V139 

It  takes  too  long  to  process  the  paperwork  needed  to  fill  va¬ 
cancies  here. 

V140 

Supervisors  here  cooperate  with  each  other  for  the  attain¬ 
ment  of  DS’s  goals. 

V142 

In  DS,  my  organization  recognizes  supervisors  who  take  the 
time  to  develop  their  subordinates’  knowledge,  skills,  and 
abilities. 

V143 

The  personnel  department  here  provides  line  management 
with  valuable  support  services. 

V149 

Supervisors  in  DS  feel  their  ability  to  manage  is  restricted  by 
unnecessary  personnel  rules  and  regulations. 

.720 

[.637] 

OPM31B 

TRAINING  OPPORTUNITIES 

V035 

DS  gives  me  adequate  training  to  do  my  job  well. 

V092 

I  am  given  the  opportunities  I  want  to  participate  in  training 
programs. 

V091 

I  am  satisfied  with  the  chances  I  have  to  team  new  things  on 
my  job. 

NOTE:  Scales  whose  codes  end  in  a  letter  are  modified  from  the  original  0PM 
scales;  added  items  are  italicized. 
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reversed  responses  are  noted  with  asterisks  (e.g.,  “*•  V007,  VlOl  re¬ 
versed”). 

Our  Alpha  coefficients  are  generally  comparable  to  those  found  in 
previous  research,®  and  the  analysis  generally  supports  the  scale 
construction  proposed  by  0PM.  The  noteworthy  exceptions  are  the 
Internal  Equity  (OPM09)  and  Locking  In  (0PM 13)  scales,  which 
yielded  Alpha  coefficients  below  .5,  and,  to  a  lesser  extent,  the 
Organizational  Influence  (0PM 16)  and  Other  Work  Groups  (OPM25) 
scales,  which  yielded  coefficients  between  .5  and  .6.  Accordingly,  the 
items  composing  these  four  scales  were  analyzed  individually.  The 
anomalous  result  for  the  Internal  Equity  scale  probably  is  at¬ 
tributable  to  the  wording  of  question  87,  which  refers  to  “other  direc¬ 
torates”  rather  than  to  “DS”  as  does  question  48.  The  basis  of  the  re¬ 
sults  for  the  three  other  scales  is  not  clear. 

Further  correlational  and  factor  analyses  were  performed  to  define 
additional  scales  for  survey  questions  that  appeared  to  assess  similar 
attitudes  but  had  not  been  grouped  together  by  0PM.  The  questions 
included  both  0PM  items  and  those  developed  by  RAND  for  PACER 
SHARE.  As  can  be  seen  in  Table  8,  the  “Pay  Determinants,”  “Union 
Satisfaction,”  “Organizational  Involvement,”  “Satisfaction  with 
Supervision/Work  Unit,”  and  “Supervisors’  Classification  Satis¬ 
faction”  groupings  yielded  large  Alpha  coefficients,  supporting  the 
combination  of  the  grouped  items.  Thus,  they  were  treated  as  scales 
in  the  PACER  SHARE  evaluation.  The  remaining  groupings  yielded 
much  smaller  Alpha  coefficients,  indicating  that  the  responses  to  the 
combined  items  were  more  independent  of  each  other.  Consequently, 
the  items  were  analyzed  individually  rather  than  combined  into 
scales. 

Evaluative  Analysis.  The  primary  survey  evaluation  consisted  of 
computing  and  comparing  the  responses  for  each  scale  or  individual 
item  at  Sacramento  ALC  and  the  comparison  sites.  For  reference 
purposes,  a  summary  analysis  was  made  to  determine  the  mean  re¬ 
sponse  by  site  for  each  item  and  scale.  For  scales,  each  participant’s 
response  was  itself  the  mean  of  his  or  her  responses  to  the  component 
scale  items  (following  any  necessary  polarity  reversals).  An 
individual’s  scale  response  was  not  counted  if  a  response  to  any  of  the 
component  items  was  missing.  (For  items,  scales,  and  sites,  missing 
response  rates  ran  from  0  to  8  percent;  rates  of  1  to  3  percent  were 
typical.)  Because  supervisors  were  disproportionately  represented  in 


^For  example,  in  the  evaluation  of  the  0PM  demonstration  at  the  Naval  Ocean 
Systems  Center. 
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Table  8 

Additional  Variable  Groups:  Questions  and  Alpha  Coefficients 


Alpha  Scale  Code  Variable  Group  Name  and  Question 

.877  PAYDETRM  Pay  Determinants  (How  important  is  each  in  determining 
your  pay?) 

VI 14  The  quality  of  your  job  performance? 

V115  The  quality  of  your  work  unit’s  performance? 

V116  The  amount  of  responsibility  on  your  job? 

.862  UNION  SAT  Union  Satisfaction 

V046  In  general,  I  like  the  way  the  union  handles  things. 

(How  satisfied  are  you  with  the  efforts  yoiu:  union  has 
made  to  get  each  of  the  following  outcomes  for  its 
members?) 

V107  More  meaningful  work  for  members? 

V108  Fairer  job  classifications? 

V109  Fairer  promotion  policies? 

VI 10  How  satisfied  are  you  with  the  success  your  union  has  in 

bargaining  non-wage  issues? 

.814  ORGINVOL  Organizational  Involvement 

VOlO  What  happens  to  DS  is  really  important  to  me. 

V016  To  help  DS,  it  is  necessary  that  I  think  of  ways  to  help 
other  sections,  branches,  or  divisions  do  their  jobs. 

V042  It  is  necessary  for  DS  to  minimize  costs  and  maximize 
performance. 

V044  It  is  necessary  for  everyone  in  DS  to  help  support  other 
directorates  such  as  Maintenance. 

V053  To  help  DS,  it  is  necessary  that  I  think  of  ways  to  help  my 
section  do  its  job. 

V062  I  am  personally  responsible  for  helping  DS  improve  its 
performance. 

V072  For  DS  to  do  its  mission  well,  it  is  necessary  for  me  per¬ 
sonally  to  do  a  good  job. 

V073  If  I  have  ideas  on  how  people  in  DS  coidd  improve  their 
work,  I  should  tell  them. 

V080  It  is  necessary  for  DS  to  maintain  high  work  quality  and 
timeliness. 

V096  I  can  save  money  for  DS  by  working  harder  or  better. 

V097  I  have  ideas  about  how  1  could  do  a  better  job  for  DS. 

.867  SUPVNUNT  Satisfaction  with  Supervision/Work  Unit 

V016  My  supervisor  encourages  subordinates  to  participate  in 
important  decisions. 

V021  My  supervisor  gives  me  adequate  information  on  how  well 
I  am  performing. 

V023  My  supervisor  has  strong  technical  skills. 

V028  My  supervisor  demands  that  people  give  their  best  effort. 

V031  My  supervisor  works  well  with  people. 

V034  My  supervisor  is  interested  in  my  opinion  on  how  to  im¬ 
prove  things  in  DS. 

V039  My  supervisor  keeps  informed  about  the  way  subordi¬ 
nates  think  and  feel  about  things. 
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Table  8— continued 


Alpha 

Scale  Code 

Variable  Group  Name  and  Question 

.723 

CLASSSAT 

Supervisors’  Classification  Satisfaction 
•*  V146  reversed 

V032 

All  in  all,  I  am  satisfied  with  the  position  classification 
procedures  in  DS. 

V146 

I  have  to  devote  too  mtich  time  to  position  classification. 

V148 

I  have  enough  authority  to  influence  claaaification  deci¬ 
sions. 

V069 

In  DS,  employees  receive  equal  pay  for  equal  work. 

V134 

In  DS  jobs  are  classified  fairly  and  accurately. 

.600 

Union-Management  Relations 
••  V003,  V098  reversed 

V003 

The  union  and  management  are  hostile  toward  each 
other. 

V093 

Management  and  the  union  are  willing  to  try  solutions 
that  haven’t  been  tried  before. 

V098 

Management  is  only  willing  to  negotiate  about  a  few  spe¬ 
cific  issues. 

.316 

Supervisors’  Perceptions  of  Grading  Criteria 

V129 

My  pay  is  based  partly  on  the  performance  of  the  workers 

I  supervise. 

V136 

The  criteria  used  to  grade  supervisory  positions  in  DS  are 
fair. 

V138 

My  pay  level  is  based  partly  on  the  number  and  grades  of 
the  people  I  supervise. 

.632 

Supervisors’  Willingness  to  Recommend  Staff  Reductions 

V130 

’The  work  I  am  responsible  for  supervising  probably  could 
be  done  with  fewer  employees. 

V144 

’The  work  I  am  responsible  for  supervising  probably  could 
be  done  with  fewer  mid-level  supervisors. 

the  comparison  site  samples,  means  were  computed  separately  for 
supervisors  and  nonsupervisors. 

The  main  evaluation  followed  the  summary  analysis.  It  consisted  of  a 
series  of  ordinary  least-squares  (OLS)  multivariate  regressions  on  the 
responses  of  the  individual  survey  participants.  The  regression  speci¬ 
fication — see  N-3404-FMP,  App.  B — permitted  responses  to  each  item 
and  scale  at  Sacramento  to  be  compared  with  the  responses  to  the 
same  questions  across  the  four  other  ALCs  taken  together 
(unweighted)  and  distinguished  baseline,  year-one,  and  year-two  out¬ 
comes.  Results  were  pooled  for  the  other  ALCs  to  reduce  the  effect  of 
regional  and  idiosyncratic  differences  among  ALCs,  and  each  partici- 


38 


pant’s  response  was  weighted  equally.®  We  chose  this  approach  be¬ 
cause  the  numbers  of  individuals  representing  each  ALC  are  both 
large  and  fairly  uniform  and  because  we  wish  to  compare  the 
Sacramento  results  with  those  for  all  other  DS  employees. 

Although  the  DSs  at  all  of  the  ALCs  perform  similar  missions,  differ¬ 
ences  in  staffing  and  sampling  for  the  survey  could  affect  the  survey 
responses.  The  regression  analysis  controlled  for  supervisory  status 
because  only  a  sample  of  the  nonsupervisors  is  selected  for  survey  at 
the  comparison  sites,  yielding  a  higher  concentration  of  supervisors, 
whose  attitudes  differ  from  those  of  nonsupervisors.  The  analysis 
also  controlled  for  differences  in  the  background  and  demographic 
characteristics  of  the  survey  respondents  at  the  different  ALCs  by  in¬ 
cluding  variables  for  tenure  at  DS,  age,  sex,  education,  and  ethnicity 
(see  Table  9).  This  control  is  necessary  because  demographic  and 
other  background  characteristics  could  have  implications  for  individ¬ 
ual  attitudes  pertaining  to  economic  incentives  or  organizational  cul- 
ture^^  and  because  those  background  characteristics  vary  with  site 
and  possibly  over  time  and  survey  sample.  In  fact,  all  but  a  few  of  the 
variables  listed  in  Table  9  showed  statistically  significant  correlations 
with  responses  to  at  least  half  our  items  and  scales,  according  to  the 
survey.^®  Variation  with  site  is  obvious  from  Table  9;  ethnicity  and 


^As  was  true  at  baseline,  the  overall  survey  response  rate  for  Wamer-Robins  was 
lower  than  for  the  other  ALCs  in  the  comparison  group.  To  be  conservative,  however, 
we  again  chose  to  give  the  same  weight  to  each  Wamer-Robins  respondent  as  to  those 
at  the  three  other  comparison  ALCs,  rather  than  giving  them  extra  weight  to  adjust  for 
their  lower  response  rate.  As  a  practical  matter,  it  may  be  noted  that  such  a  weighting 
atijustment  would  result  in  only  a  six  percentage  point  increase  in  the  proportion  of  the 
comparison  group  represented  by  Wamer-Robins.  Given  the  similarity  of  the  results 
across  the  ^Xs,  in  which  differences  in  mean  scale  scores  are  generally  less  than 
three-tenths  of  a  point,  the  adjustment  would  be  trivial,  rarely  amounting  to  more  than 
one  or  two  one-hundredths  of  a  point  on  the  five-point  scales.  Similarly,  even  if  the  at¬ 
titudes  of  the  nonparticipants  varied  to  some  extent  from  the  respondents’,  the  cross- 
ALC  similarity,  high  participation  rate,  and  personnel  record  information  imply  that 
the  effect  of  such  a  difference  on  the  survey  results  would  be  minimal. 

l®Average  supervisor  responses  differed  significantly  from  the  nonsupervisory 
averages  (p  <  .05)  for  over  90  percent  of  the  items  and  scales  measured.  (See  App.  B  in 
N-3404-FMP.) 

^^It  can  be  argued  that  changes  in  attitudes  attributable  to  changes  in  the  com¬ 
position  of  the  work  force  during  the  demonstration  are,  after  all,  also  attributable  to 
the  demonstration.  We  do  not  dispute  that  argument  here.  Our  rationale  is  simply 
that  changes  in  attitudes  among  the  same  or  similar  individuals  are  different  from 
changes  due  to  shifts  in  the  demographic  composition  of  the  work  force,  and  that  the 
regression  analysis  allows  us  to  disentangle  and  assess  each  of  these  effects. 

l^The  exceptions;  number  of  years  under  current  supervisor,  number  of  years  as  a 
federal  employee,  and  "other”  ethnicity.  Blue-/white-collar  status,  pay  grade,  and  ed¬ 
ucation  (along  with  supervisory  status)  exhibited  especially  high  frequencies  of  signifi¬ 
cant  correlation  (over  80  percent). 
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education  level  in  particular  vary  widely.  Variation  with  time  is  pos¬ 
sible  because  different  individuals  will  be  surveyed  at  different  points 
during  the  demonstration  period.  Part  of  that  variation  stems  from 
the  normal  ebb  and  flow  of  employees;  some  individuals  will  leave 
each  site  and  others  will  join.  Another  part  stems  from  variation  in 
sampling;  for  nonsupervisors  at  the  comparison  sites,  each  survey  is 
administered  only  to  a  randomly  selected  sample,  not  the  entire  popu¬ 
lation.  (And  because  the  surveys  are  filled  out  anonymously  to  en¬ 
courage  candor,  establishing  a  longitudinal  panel  is  not  possible.) 
Furthermore,  if  the  demonstration  has  the  desired  effects  on  produc¬ 
tivity  at  Sacramento,  we  would  expect  many  of  the  positions  vacated 
by  natural  attrition  to  be  left  unfilled.  This  is  likely  to  affect  the  de¬ 
mographic  and  background  characteristics  of  the  remaining  work 
force.  Moreover,  the  persons  attracted  to  those  positions  that  are 
filled  may  differ  systematically  from  persons  attracted  to  such  posi¬ 
tions  before  the  demonstration.  The  regression  specification  allows  us 
to  compare  attitudes  across  sites  and  years  while  accounting  for  the 
effects  of  such  differences  in  work  force  composition. 

Sources  for  Personnel  System  Measures 

Like  the  survey  measures,  the  personnel  system  measures  are  used  to 
evaluate  changes  in  organizational  flexibility  and  quality  of  work  life. 
They  address  such  outcomes  as: 

•  Has  the  skill  base  of  the  work  force  been  expanded? 

•  Are  pay  opportunities  under  the  demonstration  being  maintained 
or  improved,  particularly  for  junior-level  personnel  (while  holding 
the  total  wage  bill  constant)?  Is  pay  inversion  between  supervisors 
and  nonsupervisors  being  avoided? 

•  Are  crossovers  from  white-collar  to  blue-collar  positions  (and  vice- 
versa)  being  avoided? 

•  Is  the  demonstration  bringing  about  changes  in  the  percentage  or 
distribution  of  supervisors? 

•  Is  the  combination  of  interventions  reducing  turnover?  Is  this 
more  true  of  some  segments  of  the  work  force  than  others?  Has  the 
percentage  of  the  career  force  been  maintained? 

The  analysis  uses  calendar  year  1987  as  a  baseline,  April  1988- 
March  1989  for  year  one,  and  April  1989-March  1990  for  year  two.  It 
is  designed  to  detect  differences  between  Sacramento  and  the  compar¬ 
ison  ALCs,  taken  together.  Results  are  based  on  analysis  of  person- 
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nel-record  information  for  all  DS  employees.  Each  observation 
(worker’s  score  on  a  given  outcome  measure)  is  weighted  equally;  that 
is,  the  results  are  not  weighted  by  site.  Crossover  and  turnover  re¬ 
sults  are  analyzed  by  tests  for  differences  in  proportions.  All  other 
results  are  based  on  a  series  of  OLS  regression  analyses,  which  in¬ 
clude  terms  to  distinguish  the  period  (baseline,  year  one,  year  two) 
and  source  (Sacramento,  other  ALC)  of  the  observation.  Supporting 
data  and  analyses  are  provided  in  App.  C  of  N-3404-FMP. 

Personnel  measures  fall  into  one  of  three  categories,  distinguished  by 
their  source  and  reliability;  the  Work  Force  Database,  individual 
ALC  records,  and  0PM  survey  data.  Most  of  the  personnel  system 
measures,  listed  in  Tables  3  and  4,  come  from  the  WFDB.  These  data 
are  tabulated  from  automated  records  maintained  on  the  civilian 
work  force  by  the  Directorate  of  Civilian  Personnel  at  Wright- 
Patterson  Air  Force  Base.  These  are  the  most  complete  and  reliable 
records.  They  have  been  maintained  for  many  years,  and  our  own  in¬ 
spections  of  them  gave  us  no  reason  to  doubt  the  Air  Force’s  reliability 
claims.  Two  measures,  identified  as  “personnel  records”  in  Tables  3 
and  4,  must  be  tabulated  from  records  that  are  maintained  at  the 
individual  ALCs  using  special  audits.  Finally,  a  third  category  of 
measures  must  be  constructed  from  information  collected  by  the 
OPM’s  Personnel  Office  Productivity  Analysis  (POP A)  survey,  which 
is  to  be  completed  quarterly  during  the  demonstration. 

Through  consultation  with  personnel  system  experts  at  Wright- 
Patterson  and  McClellan  Air  Force  Bases,  specifications  were  devel¬ 
oped  to  obtain  reliable  computerized  measures  from  the  WFDB. 
Obtaining  reliable  data  for  the  two  smaller  sets  of  measures  (the  per¬ 
sonnel  record  and  POPA  measures)  has  proven  more  difficult.  Most 
of  the  record  information  still  is  being  compiled.  Moreover,  analysis 
of  the  preliminary  POPA  data  raised  a  number  of  reliability  ques¬ 
tions.  As  a  result,  the  POPA  data  are  still  being  compiled  and  veri¬ 
fied.  For  these  reasons,  the  results  given  in  Sec.  4  center  on  the 
WFDB  measures’  analyses.  Results  for  the  other  personnel  measures 
will  be  reported  when  the  required  information  becomes  available. 


MEASURING  QUALITY  AND  TIMELINESS  OF  WORK 
Measvires 

The  quality  and  timeliness  of  work  are  measured  using  data  provided 
by  the  Quality  (DSQ)  and  Management  (DSM)  Divisions  of  DS.  The 
analysis  focuses  on  variables  whose  routine  measurement  has  been 
mandated  by  the  Air  Force  Logistics  Command.  This  procedure 
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guarantees  that  comparable  measures  cover  a  broad  range  of  func¬ 
tions,  are  both  quantifiable  and  routinely  evaluated,  and  are  available 
for  all  five  ALCs. 

The  measures  cover  the  range  of  work  performed  within  DS,  including 
supply;  preservation,  packaging,  and  packing  of  materiel;  inspection 
of  materiel  processing  and  receiving;  transportation;  and  inventory 
(see  Tables  10  and  ID.^^  They  include  reports  of  discrepancies,  which 
reflect  errors  in  receipts  from  outside  DS  or  in  shipments  made  from 
DS;  measures  of  timeliness  in  accomplishing  tasks;  and  indicators  of 
shipping  support. 

Results  for  each  quality/timeliness  indicator  are  analyzed  using  mul¬ 
tivariate  regression.  The  analysis  tests  for  differences  in  the  propor¬ 
tion  of  errors  (or  proportion  of  actions  within  applicable  standards)  at 
the  Sacramento  AL(^  versus  the  comparison  sites  and  for  changes  in 
these  differences  over  time,  using  monthly  observations.  The  regres¬ 
sion  specification  includes  terms  to  distinguish  the  period  and  source 
of  the  outcome  (as  described  for  the  personnel  analyses  in  the  previ¬ 
ous  subsection). 

Weighting  and  Comparison  Issues 

In  the  attitude  and  personnel  system  analyses  discussed  earlier,  the 
results  for  each  individual  were  given  equal  weight,  rather  than  com¬ 
puting  separate  means  for  each  of  the  comparison  ALCs  and  then  av¬ 
eraging  those  means.  For  the  work  quality  analysis,  there  were  two 
issues  of  weighting  and  comparison  to  consider.  The  first  dealt  with 
the  weight  to  be  given  to  the  observations  for  each  ALC,  which,  in  this 
case,  consist  of  quality  rates  reported  in  a  given  month.  We  had  to  be 
careful  to  medie  sure  that  if  quality  improved  in  a  particular  work 
area,  the  periodicity  with  which  the  associated  DSQ  indicator  was  as¬ 
sessed  was  not  reduced.  If  this  pattern  were  shown  by  the  baseline 
data,  then  the  quality  of  work  for  the  comparison  group  would  be  un¬ 
derstated  by  equal  weighting  of  each  observation,  because  the  ALCs 
with  the  best  quality  would  have  fewer  observations. 

Second,  to  allow  a  sufficient  number  of  monthly  observations  for 
meaningful  baseline  data,  the  baseline  period  was  defined  as  1985  to 
1987.  The  plan  to  have  the  baseline  cover  other  years  in  addition  to 


^^Although  there  were  11  DSQ  measures  collected  command-wide  in  year  one  (of 
which  eight  had  sufficient  observations  to  be  analyzed),  directed  collection  was  discon¬ 
tinued  during  year  two.  We  were  able  to  evaluate  the  six  DS  measures  shown  in  Table 
10  nonetheless,  but  year-two  data  were  available  only  for  Sacramento. 
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Table  10 

Measures  of  Work  Accuracy 


Supply 

BL7:  Controlled  Exceptions — A  list  of  transactions  rejected  during  computer  in¬ 

put  is  checked  to  see  that  all  such  transactions  have  been  processed  accu¬ 
rately  and  in  a  timely  manner. 

Preservation,  Packaging,  and  Packing  of  Materiel 

PM:  Packing  Process — ^Before  sealing  a  shipping  container,  the  line  item  and  its 

container  are  checked  for  tagging,  quantity,  misselection,  etc.,  and  the  ac¬ 
companying  documentation  is  checked  for  accuracy  of  record. 

Materiel  Processing  and  Receiving  Inspection 

RL2:  Inspection — ^After  a  receipt  for  materiel  is  released  for  dispatch,  the  receipt 

and  its  accompanying  document  are  checked  to  see  that  they  accurately  re¬ 
port  the  identity  of  the  materiel,  its  condition,  tagging,  etc. 

RL5;  Tailgate  Date  Accuracy— When  materiel  is  off-loaded  from  the  carrier,  its 

documentation  and  asscxiated  computer  records  are  checked  to  see  that  the 
off-loading  date  is  recorded  accurately. 


Inventory 

VLl:  Location  Audit  Program  Survey — Locator  cards,  location  change  notices, 

and  physical  location  of  materiel  are  compared  as  a  check  on  the  accuracy  of 
this  Burve/s  examination  of  record-location  compatibility. 

VL3;  Physical  Count  of  Noncontrolled  Items — The  records  from  the  count  are 
checked  against  materiel  locations. 

Reports  of  Discrepancies  (RODs)  received:  reports  received  of  incorrect  shipments 
from  DS. 

Reports  of  Discrepancies  (RODs)  initiated:  reports  sent  out  notifying  senders  of 
incorrect  shipments  received  into  DS. 


Table  11 

Measures  of  Receiving  Timeliness  and  Shipping  Support 

Percentage  of  items  for  which  receiving  documents  are  posted  within  one  day 
Percentage  of  receipts  binned  within  two  days 
Percentage  of  high-priority  requisitions  (lower  is  better) 

Percentage  of  denials  of  issues  due  to  unavailability  of  items  originally  believed  to  be  in 
stock  (lower  is  better) 
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1987  raised  a  second  consideration:  verification  that  the  pattern  of 
results  at  Sacramento  for  1987  relative  to  the  results  at  other  ALCs 
was  similar  to  the  1985-1986  pattern  of  results  for  the  two  groups. 

Both  issues  pertain  to  the  DSQ  data.  The  number  of  observations  per 
ALC  is  considerably  less  variable  for  the  DSM  data,  so  only  the  sec¬ 
ond  issue  is  relevant  there. 

To  investigate  these  issues,  multivariate  regression  analyses  were 
performed  on  the  DSQ  and  DSM  data  for  1985  to  1987.^^  Overall,  the 
analyses  support  the  use  of  equal  weighting  for  the  monthly  ob¬ 
servations  and  the  combination  of  the  1987  observations  with  those 
for  the  earlier  period. 

First,  the  DSQ  data  do  not  show  inverse  correlation  between  quality 
of  work  and  frequency  of  measurement.  Of  the  13  indicators  as¬ 
sessed,  two  (RODs)  had  identical  numbers  of  observations  for  the 
comparison  ALCs.  Among  the  remaining  11,  the  (correlational)  rela¬ 
tionship  between  the  number  of  observations  for  the  ALC  and  the 
quality  level  for  the  ALC  was  evenly  split.  In  six  instances  ALCs  with 
better  quality  rates  had  fewer  observations,  and  in  five  instances  they 
had  more  observations. 

Second,  the  pattern  of  results  for  the  Sacramento  ALC  in  relation  to 
the  comparison  group  did  not  differ  systematically  between  the  1985- 
1986  period  and  1987.  For  11  of  the  13  baseline  DSQ  measures,  the 
Sacramento  data  show  the  same  relationship  to  the  comparison  group 
data  across  the  two  periods.^®  There  are  six  DSM  baseline  measures. 
In  all  six  cases,  the  pattern  of  results  for  Sacramento  relative  to  the 
comparison  ALCs  is  the  same  across  the  two  time  periods.*® 

PRESENTATION  OF  RESULTS 

The  next  four  sections  present  the  results  of  our  analyses.  Findings 
on  organizational  flexibility  and  quality  of  work  life  are  discussed  in 
two  separate  sections:  Section  3  is  based  on  the  attitude  survey  re¬ 
sults  and  Sec.  4  on  the  personnel  measure  results.  Section  5  covers 
findings  on  work  quality.  Section  6  describes  the  results  of  our  cost 
savings  analysis.  Our  conclusions  are  discussed  in  Sec.  7. 


*^Appendix  D  in  N-3146-FMP  gives  the  regression  results  and  annual  rates  for  each 
measure  over  the  three-year  period. 

t^In  terms  of  the  sign  and  significance  level  of  the  regression  coefficients  for 
Sacramento  relative  to  those  for  the  comparison  group,  as  shown  in  Table  D.l  of 
N-3146-FMP.  The  exceptions  are  PL4  and  SL4. 
l6See  Table  D.2  of  N-3146-FMP. 


3.  ORGANIZATIONAL  FLEXIBILITY  AND 
QUALITY  OF  WORK  LIFE;  ATTITUDE 
SURVEY  RESULTS 


In  the  preceding  section,  we  discussed  the  methods  and  data  we  use 
to  evaluate  the  outcomes  of  the  PACER  SHARE  Demonstration 
Project.  We  now  present  the  results  of  the  second-year  evaluation 
and  contrast  them  with  those  at  baseline  (i.e.,  predemonstration).  As 
described  in  Sec.  2,  outcomes  will  be  evaluated  using  four  types  of 
measures;  (1)  survey  results,  (2)  personnel  system  records,  (3)  work 
quality  indicators,  and  (4)  measures  of  costs  and  production.  We  be¬ 
gin  with  results  from  the  survey  analysis.  Here,  the  baseline  period 
corresponds  to  the  period  of  administration  of  the  baseline  surveys 
between  November  1987  and  March  1988;  the  year-two  surveys  were 
given  in  March  1990. 

The  survey  is  an  important  tool  for  evaluating  the  effectiveness  of  the 
PACER  SHARE  interventions  in  achieving  the  demonstration’s  goals 
of  improving  the  quality  of  work  life  and  organizational  flexibility. 
The  baseline  suivey  demonstrated  substantial  variation  in  attitudes 
toward  the  work  environment,  depending  on  the  specific  aspects  mea¬ 
sured.  But  without  question,  the  primary  area  of  disapproval  con¬ 
cerned  pay  and  promotion  practices  (i.e.,  extrinsic  rewards). 
Measures  of  the  organizational  climate  within  DS  also  refiected  unfa¬ 
vorable  ratings.  In  contrast,  measures  of  job  and  intrinsic  satisfaction 
were  either  generally  favorable  or  included  some  favorably  rated 
items. 

Baseline  attitudes  toward  existing  conditions  and  personnel  practices 
generally  were  less  favorable  at  Sacramento  ALC  than  in  the  compar¬ 
ison  group.  This  could  reflect  a  long-standing  difference  that  existed 
well  before  PACER  SHARE.  However,  evidence  suggests  that  satis¬ 
faction  with  existing  conditions  and  practices  decreased  as  PACER 
SHARE  was  developed  and  its  details  and  purposes  were  explained  to 
the  work  force.  ^ 

The  survey  analysis  for  year  two  is  intended  to  reveal  changes  in  atti¬ 
tudes  at  Sacramento  (relative  to  the  comparison  group)  toward  the 
quality  of  work  life  and  organizational  flexibility.  We  found  that  atti¬ 
tudes  toward  pay-related  attributes  have  trended  primarily  down- 


iSee  R-3753-FMP. 
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ward  at  Sacramento  (relative  to  the  comparison  group).  This  outcome 
appears  to  be  attributable  to  concerns  about  the  way  pay  and  promo¬ 
tion  are  handled  under  the  revised  base  pay  determination  proce¬ 
dures.  Attitudes  toward  nonpay-related  veudables  generally 
improved.  Moreover,  when  attitudes  toward  pay  are  controlled  for, 
attitudes  toward  other  aspects  of  work  life  and  organizational  flexibil¬ 
ity  show  substantial  improvement  relative  to  those  of  the  comparison 
group. 

ATTITUDE  MEASURES  IN  THE  EVALUATION  MODEL 

The  evaluation  model  for  PACER  SHARE  was  shown  in  Sec.  2.  The 
model  includes  numerous  attitudinal  measures  designed  to  capture 
changes  in  attitudes  during  the  demonstration.  Analyses  compare 
the  extent  of  such  changes  at  Sacramento  with  changes  at  the  other 
ALCs  during  the  same  time  period.  The  next  four  tables  specify  the 
particular  scales  and  individual  questions  analyzed  for  each  instance 
in  which  an  attitude  measure  was  listed  in  Table  3.  The  order  of  pre¬ 
sentation  and  labeling  are  the  same.  For  example,  for  intervention 
T,”  job  series  and  grade  consolidation,  expected  effect  “A”  refers  to  a 
simplified  job  classification  process.  Measure  “5”  for  lA  consists  of 
employee  perceptions  of  the  classification  process.  Specifically,  mea¬ 
sure  5  consists  of  supervisors’  responses  to  scale  CLASSSAT  (a  super¬ 
visory  scale)  and  nonsupervisors’  responses  to  questions  32  and  59 
(the  two  questions  in  CLASSSAT  also  answered  by  nonsupervisors). 

For  each  scale  and  variable  in  the  four  tables,  several  types  of  results 
may  be  seen.  The  first  column  of  numbers  indicates  the  mean 
(average)  of  the  survey  participants’  responses  for  the  attitude  mea¬ 
sure  at  Sacramento  ALC  at  baseline. 

The  second,  third,  and  fourth  columns  in  the  four  tables  represent 
OLS  regression  coefficients.^  The  coefficient  in  column  two  expresses 
the  direction  and  extent  of  the  baseline  difference  of  the  Sacramento 
attitude  level  from  the  attitude  level  of  DS  employees  at  the  other 
ALCs,  controlling  for  differences  in  the  demogr.aphic  composition  and 
experience  of  DS  employees  at  the  different  sites.  Significant  differ¬ 
ences  between  the  attitudes  of  the  Sacramento  and  comparison  site 
DS  work  forces  (i.e.,  p  <  .05)  are  indicated  by  asterisks.  'This  column 
thus  tests  the  hypothesis  that  the  predemonstration  attitude  ex¬ 
pressed  at  Sacramento  is  the  same  as  the  attitude  expressed  at  the 
other  ALCs. 


^The  regression  coeflicients  are  taken  from  App.  B  of  N-3404-FMP,  which  also  dis¬ 
cusses  the  terms  in  the  regression  model. 
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The  first  two  columns  give  a  picture  of  the  attitudes  of  DS  employees 
at  Sacramento  and  in  the  comparison  group  at  baseline.  Attitude 
levels,  however,  are  not  the  focus  of  this  evaluation.  The  goal  of  the 
demonstration  is  to  improve  quality  of  work  life  and  organizational 
flexibility  (among  other  things),  so  our  emphasis  is  on  change. 
Change  is  taken  up  in  columns  three  and  four. 

Column  three  (in  the  four  tables)  indicates  the  estimated  change  in 
the  mean  score  for  the  scale  or  question  at  the  comparison  sites  after 
the  first  two  years  of  the  demonstration,  as  assessed  by  the  year-two 
survey  and  compared  to  their  mean  score  at  baseline.  The  hypothesis 
tested — and  the  expectation,  given  nonparticipation  in  PACER 
SHARE — is  that  the  attitude  expressed  in  the  comparison  group  in 
year  two  is  the  same  as  that  expressed  there  at  baseline. 

Column  four  shows  the  estimated  second-year  change  in  mean  score 
at  Sacramento.  The  difference  from  the  change  in  mean  score  for  that 
scale  or  variable  at  the  other  sites  is  obtained  by  subtracting  column 
three  from  column  four.  This  is  the  key  number,  for  it  tests  the  hy¬ 
pothesis  that  the  change  in  attitude  at  Sacramento  is  the  same  as  the 
change  in  attitude  in  the  comparison  group,  that  is,  that  the  demon¬ 
stration  had  no  effect  on  the  attitude. 

For  example,  for  measure  I.A.5,  scale  CLASSSAT  assesses  supervi¬ 
sors’  attitudes  toward  the  job  classification  process.  The  results  in 
Table  12  indicate  that  at  Sacramento  ALC  the  mean  response  to  the 
items  in  CLASSSAT  was  2.56  at  baseline.  The  five-point  survey  re¬ 
sponse  scales  consisted  of  1  =  strongly  disagree,  2  =  disagree,  3  =  un¬ 
decided,  4  =  agree,  5  =  strongly  agree.  Thus,  as  can  be  seen  by  exam¬ 
ining  the  items  in  CLASSSAT  (see  Table  7),  a  mean  response  of  2.56 
indicates  that,  on  average,  Sacramento  respondents  tended  to  dis¬ 
agree  that  the  existing  classification  process  was  satisfactory. 

The  second  number  for  I.A.5  in  Table  12  is  -.28.  Because  site  is  de¬ 
noted  by  a  dummy  variable  that  takes  the  value  1  for  Sacramento 
(and  0  otherwise),  the  coefficient  -.28  indicates  that  the  mean  re¬ 
sponse  on  CLASSSAT  at  Sacramento  at  baseline  was  .28  lower  than 
at  the  other  ALCs,  after  adjusting  for  differences  in  the  composition  of 
the  work  forces.  In  other  words,  even  if  the  comparison  ALC  work 
force  had  the  same  demographic  and  experience  composition  as  the 
Sacramento  DS  work  force,  we  would  expect  its  mean  response  on 
CLASSSAT  to  differ  (to  equal  2.84,  i.e.,  2.56  +  .28).  The  asterisk  indi- 


Table  12 
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cates  that  the  attitude  difference  between  Sacramento  and  the  com¬ 
parison  group  is  statistically  significant.  This  result  thus  constitutes 
a  rejection  of  the  hyothesis  that  Sacramento’s  predemonstration  atti¬ 
tude  toward  the  classification  process  was  the  same  as  that  of  the 
comparison  group. 

In  column  three,  we  observe  the  change  in  people’s  attitude  two  years 
later  at  the  comparison  sites.  This  change  was  only  +.02,  indicating 
no  significant  change  (i.e.,  2.84  +  .02  =  2.86).  That  is,  we  cannot  reject 
the  h3T)Othesis  that  the  attitude  in  the  comparison  group  in  year  two 
was  the  same  as  at  baseline. 

We  are  especially  interested  in  the  unique  way  in  which  Sacramento 
attitudes  changed  as  compared  with  attitude  changes  at  the  other 
sites.  The  change  at  Sacramento  is  shown  in  column  four,  where  we 
observe  that  Sacramento  showed  a  gain  of  +.17  at  the  end  of  year  two. 
By  comparing  columns  three  and  four,  we  see  that  this  represents  an 
additional  change  of  .15  above  the  .02  at  the  other  sites  (.17  -  .02  = 
.15).  However,  the  lack  of  an  asterisk  for  the  .17  figure  indicates  that 
the  difference  in  the  change  (.15)  was  not  statistically  significant. 
Thus,  we  cannot  reject  the  hypothesis  that  the  Sacramento  second- 
year  change  in  attitude  from  baseline  was  the  same  as  the  amount  of 
change  that  occurred  at  the  other  ALCs.  We  now  turn  to  the  results, 
by  intervention.^ 


Job  Series  and  Grade  Consolidation 

Baseline.  Across  the  ALCs,  respondents  expressed  dissatisfaction* 
with  advancement  and  promotion  opportunities  and  with  the  existing 
classification  process.  Supervisors  tended  to  feel  they  did  not  have 
sufficient  flexibility  in  making  job  assignments  to  meet  workload 
changes.  Respondents  in  the  comparison  group  tended  to  be  satisfied 
with  the  intrinsic  rewards  of  their  jobs,  whereas  those  at  the 
Sacramento  ALC  were  neutral.  Sacramento  respondents  expressed 
less  satisfaction  with  existing  job  classification  procedures,  advance¬ 
ment  opportunities,  and  promotion  practices  than  their  counterparts 
at  the  other  ALCs  and  expressed  less  intrinsic  work  satisfaction  (i.e., 
were  less  likely  to  say  their  jobs  were  meaningful  or  challenging  or 


3Mean  response  values  for  each  survey  item  and  scale  are  given  for  each  ALC,  for 
supervisors  and  nonsupervisors,  in  App.  B  of  N-3404-FMP. 

^In  this  qualitative  summary,  ratings  of  about  2.8  or  less  are  taken  to  be  “low,” 
“unfavorable,”  or  indicating  "dissatisfied”  respondents;  ratings  of  about  3.2  or  more  are 
taken  to  be  "high,”  “favorable,”  or  indicating  "satisfied”  respondents;  ratings  between 
2.8  and  3.2  are  taken  as  indicative  of  an  "undecided”  or  "neutral”  response. 
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that  they  received  desired  training).  Perceived  flexibility  in  making 
job  assignments  to  meet  workload  requirements  (among  supervisors) 
did  not  differ  significantly  between  Sacramento  and  the  comparison 
group. 

Year  Two.  As  can  be  seen  in  column  three  of  Table  12,  at  the  year- 
two  follow-up  there  was  little  change  in  attitudes  at  the  comparison 
ALCs. 

Column  four  shows  that  there  were  several  unique  attitude  changes 
at  Sacramento  relative  to  those  at  the  other  ALCs.  Intrinsic  work 
satisfaction  (OPMll),  perceived  training  opportunities  (OPM31B), 
and  perceived  fairness  of  job  competition  (VI 02)  increased  signifi¬ 
cantly  at  Sacramento  relative  to  the  comparison  sites.  In  contrast, 
satisfaction  with  promotions  (OPM23)  declined  significantly. 


Revised  Base  Pay  Determination 

Baseline.  Across  the  ALCs,  almost  all  aspects  of  pay  and  promotions 
in  IIB2-IIB4  were  rated  unfavorably  (see  Table  13).  Perceived  pay 
equity  for  blue-  and  white-collar  jobs,  for  jobs  within  DS,  and  between 
DS  and  other  employers  was  lower  among  Sacramento  ALC  respon¬ 
dents.  Sacramento  employees  also  expressed  less  satisfaction  with 
pay  and  promotions.  In  contrast,  they  were  more  likely  to  believe 
that  if  they  worked  harder  they  would  be  compensated  for  their  effort 
(OPM17).  There  were  no  significant  differences  across  ALCs  in  per¬ 
ceptions  of  the  roles  of  work  quality,  responsibility,  and  length  of 
service  in  determining  pay  (PAYDETRM  and  VI 17). 

Year  Two.  For  the  most  part,  attitudes  at  the  comparison  sites  did 
not  change  much  over  the  two-year  period.  The  exception  was  that 
respondents  were  more  likely  to  agree  that  present  extrinsic  rewards 
(pay  and  promotions)  were  not  motivating  or  related  to  job  perfor¬ 
mance  (OPM17,  OPM18D,  V6).  Nonetheless,  perceived  pay  equity 
between  blue-  and  white-collar  workers  (V64)  and  overall  pay 
satisfaction  (OPM19)  improved. 

As  can  be  seen  in  column  four,  however,  Sacramento’s  attitudes  wors¬ 
ened  significantly  relative  to  other  ALCs’.  Of  the  14  measures  in 
Table  13,  nine  worsened  compared  with  the  attitudes  at  the  other 
ALCs,  whereas  only  one  (V102,  fairness  of  job  competition)  improved. 
For  example,  beliefs  that  better  work  would  lead  to  commensurately 
better  pay  (scale  OPM17)  declined  by  .23  (-.31  -  -.08  )  relative  to  the 
change  at  the  other  ALCs  during  the  first  two  years.  Similarly, 
Sacramento  employees  became  even  less  likely  than  DS  employees  at 
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the  other  ALCs  to  say  there  was  a  link  between  performance  and  re¬ 
wards  or  to  be  satisfied  with  pay,  promotions,  or  advancement  oppor¬ 
tunities.  Their  understanding  of  pay  decisions  (V12)  also  worsened. 
Based  on  the  overall  pattern  of  survey  results  and  information  re¬ 
ceived  from  Sacramento,  it  appears  that  these  changes  likely  result 
from  concerns  about  salary  advancement  and  promotions  given  pay 
banding  and  perhaps  the  elimination  of  performance  appraisals. 

Revised  Supervisory  Grading  Criteria 

Baseline.  Supervisors  generally  rated  grading  criteria  unfavorably, 
and  felt  that  their  pay  tended  to  be  based  on  the  number  and  grades 
of  the  subordinates  they  supervised  more  than  on  their  job  responsi¬ 
bilities  or  the  quality  of  their  work  (V138  and  PAYDETRM;  see  Table 
14).  Also,  they  tended  to  disagree  that  the  work  they  supervised 
could  be  handled  with  fewer  staff.  Supervisors’  perceptions  of  the  in¬ 
fluence  of  various  grading  and  pay  level  criteria  (V136,  V138,  VI 17, 
PAYDETRM)  on  their  jobs  generally  did  not  differ  between  Sacra¬ 
mento  and  the  comparison  ALCs.  The  exception  was  that  supervisors 
expressed  less  overall  satisfaction  with  the  grading  criteria  at 
Sacramento  ALC.  Supervisors  at  the  other  sites  were  much  less 
likely  than  those  at  Sacramento  to  say  that  the  work  they  supervised 
could  be  accomplished  with  fewer  subordinates  or  mid-level  supervi¬ 
sors. 

Year  Two.  At  the  other  ALCs,  supervisors’  attitudes  concerning 
these  measures  changed  during  the  two-year  period.  Although  the 
perceived  fairness  of  the  grading  criteria  (V136)  declined,  the  remain¬ 
ing  changes  were  favorable.  The  perception  that  pay  is  based  on  the 
number  and  grades  of  subordinates  decreased,  and  respondents  were 
more  willing  to  say  that  the  work  they  supervise  could  be  accom¬ 
plished  with  fewer  subordinates  and  mid-level  supervisors. 

Changes  at  Sacramento  generally  were  comparable  to  those  at  the 
comparison  sites.  Attitudes  concerning  grading  criteria  in  particular, 
however,  showed  more  positive  change  than  elsewhere.  Sacramento 
supervisors’  perception  that  their  pay  is  based  on  the  number  and 
grades  of  their  subordinates  (V138)  declined  by  an  additional  1.22 
points  relative  to  the  overall  decline  at  the  other  ALCs  (-0.12).  This 
perception  is  consistent  with  the  goals  of  the  intervention — to  base 
supervisory  pay  on  responsibilities  rather  than  on  the  number  of  peo¬ 
ple  supervised.  Similarly,  their  perception  that  supervisory  grading 
criteria  are  fair  (V136)  improved  by  .29  relative  to  the  other  ALCs. 


Table  14 

Survey  Results  Relating  to  Revised  Supervisory  Grading  Criteria  and  Productivity  Gainsharing 
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Productivity  Gainsharing 

Baseline.  Sacramento  respondents  generally  disagreed  that  cost 
savings  achieved  by  DS  through  their  efforts  would  be  shared  with 
them.  Even  so,  employees  at  Sacramento  ALC  were  much  more  likely 
to  say  that  cost  savings  would  be  shared  than  were  those  at  the  other 
ALCs. 

Year  Two.  There  was  little  change  in  attitudes  toward  this  idea 
(V36)  at  the  comparison  sites  during  the  first  two  years  of  the  demon¬ 
stration.  Attitudes  at  Sacramento  improved  significantly,  however, 
probably  from  the  gainshares  paid  in  the  second  year.  The  average 
rating  on  this  measure  rose  to  3.22  (2.75  +  .47),  reflecting  agreement 
for  the  first  time  with  the  notion  that  organizational  savings  would  be 
shared.  There  was  strong  disagreement  with  this  notion  at  the  other 
sites  (a  rating  of  2.08),  where,  of  course,  there  was  no  expectation  that 
gainshares  might  be  paid. 

Combination  of  Interventions 

Baseline.  Overall,  respondents  were  (1)  undecided  about  the  useful¬ 
ness  of  quality  measures,  (2)  dissatisfied  with  staffing  flexibility,  (3) 
undecided  about  their  intention  to  leave  DS,  (4)  dissatisfied  with  the 
organizational  climate  but  personally  committed  to  DS,  (5)  somewhat 
satisfied  with  their  ability  to  control  their  work  but  dissatisfied  with 
their  control  over  DS,  (6)  dissatisfied  with  union-management  rela¬ 
tions,  (7)  unfavorable  toward  the  adequacy  of  work  group  functioning, 
(8)  satisfied  overall  with  their  jobs  and  their  meaningfulness,  (9)  dis¬ 
satisfied  with  pay  and  promotion  practices  and  undecided  about  train¬ 
ing  opportunities,  (10)  dissatisfied  with  the  help  received  from  the 
personnel  office,  and  (11)  undecided  about  the  adequacy  of  supervi¬ 
sion  (see  Table  15).  Sacramento  respondents  expressed  significantly 
less  satisfaction  in  almost  all  these  areas. 

Year  Two.  Column  three  of  Table  15  indicates  that  most  attitudes 
did  not  change  significantly  at  the  comparison  sites  during  the  two- 
year  period.  Among  those  that  did  change  for  the  better,  turnover 
intention  tended  to  decline  (V47,  V83)  and  pay  satisfaction  (0PM 19) 
increased.  On  the  down  side,  supervisors  expressed  less  overall  satis¬ 
faction  with  the  supervision  system  (OPM30),  respondents  were  less 
likely  to  say  they  knew  whether  their  work  was  satisfactory  (VI),  per¬ 
ceived  union-management  hostility  (V3)  increased,  and  pay  and  ad¬ 
vancement  opportunities  were  perceived  as  less  motivating  or  con¬ 
nected  to  job  performance. 


Survey  Results  Relating  to  the  Combination  of  Interventions 
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Regression  Coefficient 
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Overall,  the  pattern  of  changes  at  Sacramento  (relative  to  those  at  the 
other  ALCs)  depends  on  the  area  of  measurement.  Attitudes  toward 
pay  and  promotion  worsened  at  Sacramento  relative  to  the  other 
ALCs,  and  turnover  intention  increased  on  some  measures.  In  con¬ 
trast,  there  was  considerable  evidence  of  differential  improvement  in 
attitudes  concerning  nonfinancial  aspects  of  the  work  environment. 
Perceptions  concerning  the  organizational  climate  and  personal  in¬ 
volvement,  organizational  influence,  group  functioning,  supervision, 
and  Personnel  Office  helpfulness  tended  to  improve  relative  to  the 
other  ALCs,  and  intrinsic  (nonfinancial)  reward  satisfaction  also  in¬ 
creased  significantly. 

ADDITIONAL  ATTITUDE  MEASURES 

Tables  12-15  include  only  those  scales  and  questionnaire  items  per¬ 
taining  to  the  evaluation  model.  Some  of  the  variables  we  analyzed 
were  not  included  in  the  model.  We  did  not  hypothesize  a  priori  that 
these  variables  would  be  affected  by  PACER  SHARE.  We  analyzed 
them  nevertheless  because  they  are  important  indicators  of  attitudes 
toward  work,  and  any  changes  in  them  that  could  be  associated  with 
the  demonstration  would  provide  useful  information.  The  scales, 
questions,  and  results  for  baseline  and  year  two  are  shown  in 
Table  16. 

Baseline 

The  upper  portion  of  the  table  shows  results  for  four  0PM  scales  in¬ 
volving  extrinsic  reward  importance,  intrinsic  reward  importance,  ex¬ 
pectancy,  and  adequacy  of  authority  among  supervisors.  The  data 
indicate  that  at  baseline  extrinsic  and  intrinsic  rewards  were  both 
rated  as  being  very  important — indeed,  extrinsic  reward  importance 
received  the  highest  mean  rating  of  any  scale — and  that  hard  work 
was  expected  to  lead  to  good  job  performance.  They  also  indicate  that 
supervisors  felt  they  needed  more  authority  in  making  decisions  con¬ 
cerning  their  subordinates.  In  each  case,  the  Sacramento  ratings 
were  lower. 

The  lower  portion  of  the  table  shows  results  for  various  other  ques¬ 
tions.  The  results  are  more  mixed.  Respondents  tended  to  feel  that 
performance  was  not  related  to  turnover  or  to  work  assignments,  new 
employees  were  not  well  qualified,  they  would  take  a  different  job  in 
DS  if  they  could,  and  they  would  be  unwilling  to  serve  on  a  union- 
management  committee.  They  were  undecided  about  the  utility  of 


for  Additional  Attitude  M 
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performance  appraisals.  In  contrast,  respondents  were  likely  to  agree 
that  they  worked  hard  and  had  the  skills  they  needed.  Supervisors 
tended  to  fetl  that  the  existing  system  enabled  them  to  improve  their 
subordinates’  performance,  and  there  was  general  agreement  that 
securing  a  “career”  position  was  important.®  Consistent  with  the 
discussion  of  Tables  12-15,  the  most  common  reasons  for  taking  a 
new  job  were  to  secure  better  pay  and  better  promotion  opportunities, 
which  were  included  as  one  of  the  three  most  important  reasons  by  77 
and  62  percent  of  the  respondents,  respectively.  Getting  more 
interesting  work  was  the  next  most  common  reason,  but  was 
mentioned  by  a  much  smaller  percentage  of  the  respondents  (38 
percent).  The  remaining  reasons  were  mentioned  by  still  smaller 
percentages. 

Generally,  Sacramento  ALC  respondents  were  less  inclined  to  agree 
with  the  statements,  indicating  less  favorable  attitudes  toward  exist¬ 
ing  conditions.  Sacramento  respondents  were  no  more  likely  to  want 
to  change  jobs  within  DS  or  to  be  willing  to  serve  on  a  union-man¬ 
agement  committee.  Also,  for  the  most  part  they  did  not  have  differ¬ 
ent  reasons  for  potential  job  turnover.  Among  12  reasons  assessed, 
only  three  differed  significantly  in  the  frequency  with  which  they 
were  rated  as  being  among  the  most  important  reasons.  Sacramento 
respondents  were  more  likely  to  say  they  would  take  a  new  job  to  get 
more  interesting  work  and  less  likely  to  say  they  would  do  so  for  bet¬ 
ter  pay  or  promotion  opportunities. 

Year  Two 

There  were  some  isolated  changes  in  attitudes  on  these  measures  at 
the  comparison  ALCs  during  the  two-year  period.  Respondents  were 
less  likely  to  agree  that  hard  work  or  innovation  led  to  good  job  per¬ 
formance  or  co-worker  recognition,  that  they  would  leave  to  get  more 
responsibility  or  better  promotion  opportunities,  or  that  annual  per¬ 
formance  appraisals  were  desirable.  They  were  more  likely  to  see 
new  employees  as  being  qualified.  Supervisors  were  less  inclined  to 
agree  they  had  sufficient  authority. 

Differences  in  the  changes  at  Sacramento  were  limited.  The  expecta¬ 
tion  that  hard  work  or  innovation  leads  to  good  performance  in¬ 
creased  relative  to  the  other  ALCs.  However,  the  perception  that 
high  performers  stay  with  DS,  that  poor  performance  would  result  in 


^Securing  a  career  position  was  significantly  more  important  to  those  without  a  ca¬ 
reer  position. 
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negative  consequences,  or  that  supervisors  have  sufficient  authority 
(among  supervisors)  declined.  Support  for  performance  appraisal  did 
not  decline  as  much  as  at  the  other  ALCs,  and  respondents  were  less 
likely  to  say  they  wanted  another  job  in  DS.  Respondents  were  more 
likely  to  say  they  would  take  a  new  job  to  get  better  pay  or  job  secu¬ 
rity  (among  DOCs),  but  they  were  less  likely  to  say  they  would  leave 
to  gain  more  interesting  work. 


ASSESSING  THE  INFLUENCE  OF  ATTITUDES  TOWARD  PAY 

As  mentioned  in  our  discussion  of  Tables  12-15,  the  pattern  of  change 
in  responses  concerning  pay  and  promotions  during  the  first  two 
years  was  different  at  Sacramento  than  at  the  other  ALCs.  These 
attitudes,  which  were  very  unfavorable  to  begin  with,  worsened 
(became  significantly  more  negative)  at  Sacramento  during  the  first 
two  years  of  PACER  SHARE  relative  to  the  comparison  sites.  We  at¬ 
tribute  this  result,  at  least  in  part,  to  concerns  about  salary  advance¬ 
ment  through  the  PACER  SHARE  pay  bands. 

This  pattern  of  responses  led  us  to  wonder  whether  people  might  be 
sufficiently  upset  about  the  pay  areas  that  there  might  be  a  carryover 
effect  on  the  responses  to  other  types  of  scales,  which,  conceptually, 
are  independent  of  the  pay  dimensions.  Such  effects  were  found  after 
year  one.  Broad  groups  of  scales  that  are  potentially  affected  are 
those  classified  as  pertaining  to  satisfaction  with  supervision  and  co¬ 
worker  interactions,  overall  work  (intrinsic)  satisfaction,  and  miscel¬ 
laneous  work  environment  perceptions.®  Thus,  we  reanalyzed  the 
data  to  control  for  such  an  influence  and  observe  the  effect,  if  any,  on 
the  results.  Specifically,  we  examined  whether  controlling  for  the 


®To  view  the  overall  pattern  of  attitudes  more  clearly,  we  performed  a  factor  analy¬ 
sis  on  the  scales  to  group  scales  that  were  conceptually  similar  into  broader  measures 
of  underlying  attitudes.  These  broader,  underlying  attitudes  are  called  attitude  factors. 
The  results  and  rationale  for  this  scale-grouping  process  into  factors  are  discussed  in 
App.  C  of  N-3257-FMP.  At  this  point,  we  want  to  note  that  our  analysis  revealed  four 
m^jor  attitude  factors  and  their  constituent  scales  (i.e.,  the  scales  that  contributed  the 
most  to  a  particular  factor): 

•  Satisfaction  with  supervision  and  co-worker  interactions 

•  Overall  work  satisfaction 

*  General  pay  satisfaction 

*  Reward  system  satisfaction 

It  would  be  meaningless  to  analyze  the  influence  of  removing  pay-performance 
linkage  perceptions  on  the  scales  comprising  the  factors  of  general  pay  satisfaction 
(third  factor  above)  and  reward  system  satisfaction  (fourth  factor  above),  since  such 
scales  concern  pay.  Thus,  we  analyzed  only  the  two  nonpay-related  factors  shown 
above  and  a  third  group  of  miscellaneous  scales  that  did  not  fall  clearly  onto  a  single 
factor. 
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perceived  link  between  pay  and  performance  (represented  by  scale 
OPM18D)  in  a  new  series  of  regression  analyses  would  have  an  im¬ 
pact  on  the  scalar  outcomes  shown  in  Tables  12-15.  “Controlling  for” 
these  perceptions  meant  inserting  the  respondent’s  score  on  OPM18D 
on  the  right-hand  side  of  the  regression  equation,  which,  in  effect, 
removed  its  effect  on  the  outcome  measures.  This  allowed  us  to  exam¬ 
ine  the  changes  in  the  mean  score  for  a  scale  independent  of  the  re¬ 
spondents’  perceptions  of  the  link  between  pay  and  performance. 

Table  17  presents  the  regression  coefficients  for  the  estimated  change 
in  Sacramento  responses  during  the  first  two  years  before  and  after 
controlling  for  OPM18D.  The  first  column  of  numbers  is  drawn  from 
column  four  of  Tables  12-15.  The  second  column  reflects  the  results 
after  controlling  for  responses  to  OPM18D.  Asterisks  indicate  that 
Sacramento’s  change  over  the  two-year  period  differed  significantly 
from  that  at  the  other  ALCs. 

As  can  be  seen  in  Table  17,  all  indicators  of  satisfaction  with  supervi¬ 
sion  and  co-worker  interactions  improved  significantly  at  Sacramento 

Table  17 

Regression  Results  for  Attitude  Scales  Controlling  for 
Perceptions  of  Pay-Performance  Link;  Year  Two 


Regression  Coefficient  for 
Year  Two  Change  at 
Sacramento 

Without  With 

Scale  Code  Measure  Control  Control 


SATISFACTION  WITH  SUPERVISION  AND  CO-WORKER  INTERACTIONS 


OPM06 

General  supervision/direction 

,18* 

.30* 

OPM07 

Group  functioning 

.15* 

.27* 

OPM14 

Open  group  process 

.24* 

.34* 

SUPVNUNT 

Satisfaction  with  supervision/work  unit 

,22* 

.34* 

OVERALL  WORK  SATISFACTION 

0PM 11 

Intrinsic  Work  Satisfaction 

.14* 

.25* 

0PM 12 

Job  Satisfaction 

.05 

.18* 

OPM08 

Intent  to  Turn  Over 

-.01 

-.12* 

MISCELLANEOUS  WORK  ENVIRONMENT  PERCEPTIONS 

0PM 15 

Organizational  Climate 

.05 

.16* 

OPM02 

Control  over  Work 

.12* 

.21* 

0PM21B 

Reconsideration/Redress 

.08 

.19* 

0PM31B 

Training  Opportunities 

.18* 

.31* 

ORGINVOL 

Organizational  Involvement 

.08* 

.12* 

NOTE;  Asterisk  indicates  the  difference  in  coefficients  between  Sacramento 
and  the  comparison  sites  is  significant  at  the  p  <  .05  level. 
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relative  to  the  comparison  sites.  Results  in  the  other  two  groups  also 
were  encouraging.  About  half  of  the  scales  showed  significant  im¬ 
provement  relative  to  the  other  ALCs,  and  none  worsened.  The  re¬ 
sults  are  even  stronger  after  controlling  for  perceptions  of  the  pay- 
performance  link.  Every  scale  increased  significantly  relative  to  the 
other  ALCs.  From  this  analysis,  we  infer  that  nonpay-related  atti¬ 
tudes  may  have  been  depressed  by  dissatisfaction  over  pay.  It  could 
be  argued  that  the  causality  relation  runs  in  the  other  direction,  but 
attitudes  toward  pay  are  generally  the  least  favorable  and  show  more 
change  than  the  others.  We  believe  that  it  is  more  likely  that  poor, 
deteriorating  attitudes  would  depress  improvement  in  other  areas 
than  that  less  unfavorable,  more  stable  attitudes  would  cause  poor 
attitudes  to  deteriorate. 

RESULTS  FOR  NEW  MEASURES 

Table  18  gives  regression  results  for  16  new  attitude  questions  added 
to  the  survey  in  year  one  at  the  request  of  the  sponsor.  The  questions 
can  be  grouped  into  four  areas:  information  use/feedback,  quality 
circles,  team  building,  and  miscellaneous.  The  factor  analysis  under¬ 
lying  this  grouping  was  discussed  in  N-3257-FMP. 

Year  One 

At  the  first-year  survey  point,  attitudes  in  these  areas  generally  were 
unfavorable  at  Sacramento.  The  exception  concerned  quality  circles, 
and  in  particular  the  opportunity  to  participate  in  them.  With  few 
exceptions,  attitudes  were  significantly  less  favorable  than  at  the 
other  ALCs,  where  they  tended  to  be  neutral  (i.e.,  in  the  3.0  range). 
Attitudes  toward  team  building  could  not  be  assessed  at  the  other 
ALCs  because  the  concept  was  implemented  at  Sacramento  alone.  As 
discussed  earlier,  it  is  likely  that  the  lower  ratings  at  Sacramento  are 
accounted  for  by  sensitization  of  the  work  force  to  alleged  deficiencies 
in  the  existing  system,  which  occurred  just  before  the  innovations 
were  implemented.  As  would  be  expected,  the  perceived  opportunity 
to  participate  in  quality  circles  was  significantly  greater  at  Sacra¬ 
mento. 

Year  Two 

There  was  little  change  in  the  attitudes  assessed  by  these  measures 
at  the  other  ALCs  between  years  one  and  two.  Only  two  of  11 
changed  significantly:  respondents  were  less  likely  to  agree  that  par- 


Table  18 
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ticipative  employees  were  the  ones  hired  or  that  they  could  partici¬ 
pate  in  quality  circles.  In  contrast,  most  of  the  measures  showed  sub¬ 
stantial  improvement  at  Sacramento.  Team-building  measures  in 
particular  showed  large  positive  shifts  in  attitude,  generally  on  the 
order  of  one-quarter  to  one-half  point  on  the  five-point  scale.  About 
half  of  the  other  scales  showed  significant  gains  relative  to  the  com¬ 
parison  sites,  and  none  worsened. 

SUMMARY 

A  broad  series  of  responses  indicates  that  perceptions  of  overall 
working  conditions  improved  significantly  during  the  first  two  years 
of  PACER  SHARE.  Satisfaction  with  supervision  and  co-worker  in¬ 
teractions,  overall  work  satisfaction  (the  meaningfulness  of  the  job), 
and  other  work  environment  perceptions  (trust  in  management,  con¬ 
trol  of  work,  training  opportunities,  organizational  involvement)  im¬ 
proved  significantly  at  Sacramento  by  the  end  of  year  two  as  com¬ 
pared  with  the  other  ALCs,  where  limited  systematic  change  occurred 
during  the  two-year  period.  In  contrast,  satisfaction  with  pay  and, 
particularly,  with  the  connection  between  job  performance  and  com¬ 
pensation  or  advancement  worsened  significantly.  Further  analysis 
indicates  that  the  pay  and  advancement  concerns  may  have  de¬ 
pressed  nonpay-related  attitudes  such  as  those  concerning  the  three 
areas  mentioned  previously.  The  implication  is  that  dissatisfaction 
with  pay  and  advancement  under  the  job  series  consolidation  and  re¬ 
vised  base  pay  determination  innovations  must  be  addressed  to  be 
certain  the  improvement  in  nonpay  areas  is  observed  fully. 

In  addition  to  their  relation  to  the  quality  of  work  life,  attitude 
changes  can  be  addressed  in  terms  of  their  consistency  with  the  goal 
of  increasing  organizational  flexibility  under  PACER  SHARE. 
Attitude  changes  generally  supported  this  goal.  For  example,  per¬ 
ceived  staffing  flexibility,  training  opportunities,  job  competition  fair¬ 
ness,  supervisory  grading  criteria  fairness,  gainsharing  of  organiza¬ 
tional  cost  savings,  and  Personnel  Office  helpfulness  increased. 
Other  perceptions  showed  no  change  relative  to  the  comparison  ALCs; 
blue-collar  versus  white-collar  pay  equity,  staffing  needs,  job  classifi¬ 
cation  satisfaction,  and  career  opportunities. 

Finally,  additional  attitude  questions  showed  increases  in  perceived 
information  exchange  in  accomplishing  day-to-day  work,  usefulness  of 
quality  circle  participation,  and  emphasis  of  team-building  concepts 
in  day-to-day  operations.  These  changes  are  consistent  with  both  bet- 
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ter  quality  of  work  life  and  improved  work  quality.  Many  of  the 
changes — especially  those  for  team  building — were  large  and  highly 
significant. 


4.  ORGANIZATIONAL  FLEXIBILITY  AND 
QUALITY  OF  WORK  LIFE:  RESULTS  FOR 
PERSONNEL  SYSTEM  MEASURES 


To  complement  the  analyses  related  to  the  quality  of  work  life  and  or¬ 
ganizational  flexibility  in  Sec.  3,  we  turn  to  personnel  system  mea¬ 
sures  in  this  section  to  provide  answers  to  questions  such  as  the  fol¬ 
lowing: 

•  Has  the  skill  base  of  the  work  force  been  expanded? 

•  Are  pay  opportunities  under  the  demonstration  being  maintained 
or  improved,  particularly  for  junior-level  personnel  (while  holding 
the  total  wage  bill  constant)?  Is  pay  inversion  between  supervisors 
and  nonsupervisors  being  avoided? 

•  Are  crossovers  from  white-collar  to  blue-collar  positions  (and  vice- 
versa)  being  avoided? 

•  Is  the  demonstration  bringing  about  changes  in  the  percentage  or 
distribution  of  supervisors? 

•  Is  the  combination  of  interventions  reducing  turnover?  Is  this 
more  true  of  some  segments  of  the  work  force  than  others?  Has  the 
percentage  of  the  career  force  been  maintained? 

We  do  not  have  results  for  simplicity  of  job  classification  or  ability  to 
fill  vacancies,  which  are  to  be  drawn  from  ALC  personnel  records  and 
OEM’s  Personnel  Office  Productivity  Analysis.  Results  for  the  miss¬ 
ing  measures  will  be  reported  later,  when  the  required  information 
becomes  available. 

We  reiterate  that  the  primary  purpose  of  the  analysis  is  to  establish 
the  predemonstration  level  for  each  measure  at  Sacramento  and  the 
comparison  sites,  and  to  compare  the  rate  of  change  for  the 
Sacramento  ALC  during  the  demonstration  with  that  for  the  other 
ALCs  (taken  together)  during  the  same  time  period.  Differential 
change  will  be  attributed  to  the  demonstration.  The  discussion  will 
cover  both  the  baseline  levels  for  the  measures  (levels  for  calendar 
year  1987)  and  any  differences  between  Sacramento  and  the  compari¬ 
son  ALCs,  taken  together.  The  tables  that  follow  also  show  results  for 
the  two-year  change  at  the  vomparison  sites  and  Sacramento,  and 
indicate  the  significance  of  the  difference  between  the  change  at 
Sacramento  and  the  pooled  comparison  sample  (with  each  observation 
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weighted  equally).  Results  are  based  on  analysis  of  personnel  record 
information  for  all  DS  employees. 

As  discussed  later,  most  of  the  results  are  based  on  a  series  of  OLS 
regression  analyses  performed  on  the  outcome  measures.  These  in¬ 
clude  predictor  variables  to  distinguish  results  for  Sacramento  base¬ 
line  (CY  1987),  year  one,  and  year  two,  and  the  comparison  group  re¬ 
sults  for  year  one  and  for  year  two  from  results  for  the  comparison 
sites  at  baseline  (the  “left  out”  group).^  Crossover  and  turnover  re¬ 
sults  represent  raw  data.  Supporting  data  and  analyses  for  all  re¬ 
sults  in  this  section  are  given  in  App.  C  of  N-3404-FMP. 

VARIABLES  AFFECTED  BY  JOB  SERIES  CONSOLIDATION 

Table  19  presents  results  concerning  the  level  of  multiple  skill  train¬ 
ing.  The  results  correspond  to  measures  I.B.2  and  I.C.l  in  the  evalu¬ 
ation  model.  (See  Tables  C.IO  and  C.ll  in  N-3404-FMP  for  related 
information.)  Recall  that  a  goal  of  job  series  consolidation  is  to  in¬ 
crease  the  organization’s  ability  to  assign  workers  to  the  positions 
where  they  are  required  to  handle  fluctuations  in  workload  without 
the  encumbrance  of  narrowly  defined  job  descriptions.  This  flexibility 
hinges  not  only  on  series  consolidation,  but  on  the  provision  of  re¬ 
quired  multiple  skill  training.  Thus,  in  support  of  this  goal  we  would 
anticipate  an  increase  in  the  average  number  of  skills  trained  per 
employee  at  Sacramento  relative  to  the  comparison  sites.  This  expec¬ 
tation  leads  to  the  testable  hypotheses  relevant  to  the  measure  in 
Table  19: 

•  The  average  number  of  skills  per  employee  at  Sacramento  is  the 
same  at  baseline  as  that  in  the  comparison  group. 

•  There  was  no  change  in  the  average  number  of  skills  per  employee 
at  the  comparison  sites  over  the  first  two  years  of  the  demonstra¬ 
tion. 

•  The  change  in  the  average  number  of  skills  per  employee  at 
Sacramento  is  the  same  over  the  first  two  years  of  the  demonstra¬ 
tion  as  that  in  the  comparison  group.  (Rejection  implies  a  greater 
increase  in  flexibility  if  Sacramento’s  change  is  larger,  less  flexibil¬ 
ity  if  it  is  smaller.) 


iThere  are  no  controls  for  background  variables  because,  unlike  the  survey  analysis, 
results  are  based  on  information  for  every  member  of  the  work  force  at  each  ALC 
during  each  time  period,  and  because  the  evaluation  design  accounts  for  differences 
among  the  ALCs  that  existed  prior  to  the  demonstration.  Moreover,  possible  work 
force  composition  changes  that  influence  the  outcomes  discussed  in  this  section,  such  as 
the  total  labor  bill,  are  relevant  to  those  outcomes. 
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Table  19 

Changes  Associated  with  Job  Series  Consolidation:  Number  of  Skills 
(measures  I.B.2  and  I.C.1  in  evaluation  model) 


Regression  Coefficient 

Measure 

Sacramento 

Baseline 

Rate 

Sacramento 
Baseline  Diff. 
from  Other  ALCs 

Year  2 
Change  for 
Other  ALCs 

Year  2 
Change  for 
Sacramento 

Average  number 
of  skills 

6.5 

-0.2* 

0.2* 

0.5* 

NOTE:  Asterisk  in  column  two  or  three  indicates  the  coefficient  is  significant  at  the 
p  <  .05  level;  in  column  four,  it  indicates  the  difference  in  the  coefficients  for  columns 
three  and  four  is  significant  at  p  <  .05. 


Baseline 

Table  19  indicates  that  the  average  number  of  skills  coded  for 
Sacramento  DS  employees — whether  DS-related  or  not — was  6.5  at 
baseline.  This  was  significantly  lower  than  the  average  at  the  other 
ALCs  (6.5  — .2  =  6.7).  The  key  question  concerns  how  this  pattern 
changed  for  DS-related  skills  during  the  first  two  years  of  PACER 
SHARE. 


Year  Two 

As  seen  in  column  three,  skill  training  increased  by  .2  skills  (one  skill 
per  five  employees)  at  the  other  ALCs  by  the  end  of  year  two.  This 
change  was  statistically  significant,  that  is,  the  skill  base  expanded  at 
the  other  ALCs.  The  change  at  Sacramento  was  .5  skills — one  skill 
per  two  employees — as  shown  in  column  four.  This  increase  was  sig¬ 
nificantly  larger  than  at  the  comparison  sites.  Although  the  baseline 
results  represent  all  coded  skills,  the  changes  for  year  two  occurred 
among  DS  employees  and  thus  represent  changes  in  DS-related  skills. 
The  results  are  consistent  with  the  expansion  of  the  skill  base  at 
Sacramento  during  the  first  two  years,  relative  to  the  incidence  of 
multiple  skill  training  at  the  other  ALCs. 

VARIABLES  AFFECTED  BY  REVISED  BASE  PAY 
DETERMINATION 

Salaries 

We  next  examine  salaries  by  pay  level  and  compare  supervisor  and 
nonsupervisor  salaries.  We  examined  salaries  by  pay  band  to  observe 
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whether  there  would  be  greater  salary  growth  at  Sacramento — exclu¬ 
sive  of  any  gainshare  payments — than  at  the  other  ALCs  as  a  result 
of  this  intervention.  In  general,  there  are  two  reasons  why  such 
salary  growth  might  occur:  (1)  pay  rates  went  up,  or  (2)  the  composi¬ 
tion  of  the  workers  in  a  specific  pay  band  group  changed,  and  hence 
the  wages  paid  within  that  group  changed.  We  also  compare  supervi¬ 
sors’  versus  nonsupervisors’  salaries  to  investigate  possible  pay  inver¬ 
sion  that  could  result  from  decoupling  the  respective  pay  schedules. 
The  “null”  hypotheses  that  apply  to  this  analysis  are  thus  as  follows: 

•  Changes  in  pay  rates  at  Sacramento  are  the  same  as  in  the  com¬ 
parison  group.  (Rejection  of  the  hypothesis  due  to  a  significant  in¬ 
crease  implies  an  unfavorable  effect  if  the  total  wage  bill  goes  up 
and,  if  it  does  not,  a  favorable  effect.) 

•  Supervisory  and  nonsupervisory  pay  rates  have  not  grown  closer  at 
Sacramento  than  they  have  in  the  comparison  group  (inversion 
implies  an  unfavorable  effect). 

The  results  are  shown  in  Table  20  and  correspond  to  measures  II  .A.  1 
and  II.C.l  in  the  evaluation  model.  (See  Tables  C.l  and  C.2  in 
N-3404-FMP.)  The  DH4,  DW4,  and  DX4  levels  are  not  shown 
because  of  the  very  small  number  of  persons  they  represent.  Table  20 
shows  the  baseline  pay  rates  (end  of  calendar  year  1987)  at 
Sacramento  (column  one),  their  differences  from  the  baseline  rates  at 
the  other  ALCs  (column  two),  and  changes  in  the  rates  of  pay  during 
the  first  two  years  for  other  ALCs  and  Sacramento  (columns  three 
and  four).  Wages  for  blue-collar  nonsupervisors  are  shown  in  dollars 
per  hour.  Salaries  for  white-collar  nonsupervisors  and  for  supervisors 
are  shown  in  thousands  of  dollars  per  year.  All  figures  represent 
nominal  dollars — dollars  uncorrected  for  inflation. 

Baseline.  Looking  first  at  the  blue-collar  nonsupervisors,  we  find 
that  the  average  hourly  wage  was  higher  at  Sacramento  than  at  the 
comparison  ALCs  for  each  pay  level.  The  reason  is  that  blue-collar 
wages  are  set  in  part  according  to  prevailing  local  wages,  which  are 
high  in  Sacramento.  The  results  for  the  white-collar  nonsupervisors 
and  for  the  supervisors  show  a  different  pattern.  The  average  annual 
salary  for  the  DW3  and  DX3  bands  did  not  differ  significantly  be¬ 
tween  Sacramento  and  the  comparison  sites.  For  the  DW2  band,  the 
average  baseline  salary  was  significantly  lower  at  Sacramento, 
whereas  it  was  higher  for  the  DWl  band.  Results  for  the  remaining 
supervisors  also  were  mixed.  The  mean  DXl  salary  was  significantly 
lower  at  Sacramento  than  for  the  comparison  group,  whereas  the  DX2 
salary  was  higher.  Overall,  then,  the  baseline  salaries  against  which 
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Table  20 

Changes  Associated  with  Revised  Base  Pay  Determination: 
Salaries  by  Pay  Band® 

(measures  IIA.1  and  ILC.l  in  evaluation  model) 


Regression  Coefficient 


Sacramento 

Sacramento 

Year  2 

Year  2 

Baseline 

Baseline  Diff. 

Change  for 

Change  at 

Pay  Band  Equates  to 

Rate 

from  Other  ALCs 

Other  ALCs 

Sacramento 

WAGES 

Blue-collar  nonsupervisors 

DHl  WGl-3 

$  9.10/hr 

0.89* 

0.59* 

-0.33* 

DH2  WG4-8 

11.81 

1.60* 

0.41* 

1.24* 

DH3  WG9-11 

13.94 

1.66* 

0.60* 

1.06* 

SALARIES 

White-collar  nonsupervisors 

DWl  GSl-4 

$15.2/yr 

0.4* 

1.0* 

-0.1* 

DW2  GS5-8 

18.7 

-0.3* 

1.7* 

3.0* 

DW3  GS9-12 

27.7 

-0.0 

2,7* 

3.2 

Supervisors 

DXl  GS5-8 

$22.0/yr 

-5.2* 

1.8* 

3.1 

DX2  GS9-12 

33,1 

1.2* 

2.4* 

3,3 

DX3  GS13-14 

47.8 

2.4 

4,8* 

4,0 

NOTE  Asterisk  in  column  two  or  three  indicates  the  coefficient  is  significant  at  the 
p  <  ,05  level;  in  column  four,  it  indicates  the  difference  in  the  coefficients  for  columns 
three  and  four  is  significant  at  p  <  .05. 

®DH  figures  represent  dollars  per  hour.  DW  and  DX  figures  indicate  annual  salary 
(in  thousands).  FWS  supervisors  were  assigned  to  DX  bands  by  converting  their  pre¬ 
demonstration  wages  to  salary  equivalents  and  placing  them  where  equally  paid  GS 
workers  were  placed. 

changes  under  PACER  SHARE  are  judged  were  higher  at  Sacramento 
for  blue-collar  nonsupervisors,  and  either  higher  or  lower— depending 
on  the  pay  band — for  white-collar  nonsupervisors  and  for  supervisory 
personnel.  White-collar  employees  at  Sacramento  in  pay  bands  1  and 
2  earned  notably  less  than  their  blue-collar  counterparts,  a  disparity 
that  revised  base  pay  determination  is  intended  to  reduce  as  the 
demonstration  proceeds.  Finally,  *he  supervisory  salaries  at  Sacra¬ 
mento  do  not  reflect  evidence  of  predemonstration  pay  inversion: 
they  are  higher  than  the  mean  salaries  for  the  comparable  white-col¬ 
lar  nonsupervisory  levels  (i.e.,  DXl  vs.  DW2  and  DX2  vs.  DW3). 

Year  Two.  Column  three  shows  the  wage  and  salary  changes  at  the 
other  ALCs  during  the  two-year  period.  As  would  be  expected  based 
on  inflation,  pay  increased  for  each  band.  As  mentioned  above,  these 
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changes  also  could  result  from  changes  in  composition  of  the  work 
force.  Changes  in  composition  are  determined  by  (1)  the  workers  who 
left  and  (2)  those  who  were  hired.  Additional  analyses — which  exam¬ 
ined  pay  changes  for  employees  present  over  the  entire  two-year  pe¬ 
riod  and,  among  those  employees,  persons  remaining  in  the  same  pay 
band  throughout — indicate,  however,  that  composition  changes  do  not 
account  for  the  pay  growth;  pay  grew  significantly  for  employees  who 
worked  at  DS  throughout  the  two-year  period. 

Column  four  indicates  the  size  of  pay  changes  at  Sacramento,  and  the 
asterisks  indicate  their  statistical  significance  relative  to  the  other 
ALCs  at  the  end  of  year  two.  We  see  that  in  about  half  the  cases,  the 
change  at  Sacramento  was  significantly  different  from  that  in  the 
comparison  group.  In  two  cases,  DHl  and  DWl,  pay  declined  signifi¬ 
cantly  relative  to  the  other  ALCs — because  of  composition  changes 
and,  in  particular,  new  hires.  Pay  grew  for  employees  who  continued 
to  work  for  DS  at  Sacramento  throughout  the  two-year  period. 
Annual  pay  grew  significantly  at  Sacramento  relative  to  the  other 
ALCs  for  the  blue-collar  nonsupervisor  pay  bands  DH2  and  DH3  and 
for  the  white-collar  nonsupervisor  pay  band  DW2.  We  attribute  the 
greater  rise  for  the  DH2  and  DH3  bands  primarily  to  actual  pay  in¬ 
creases  rather  than  to  composition  changes.  In  contrast,  the  greater 
change  for  the  DW2  band  appears  to  reflect  composition  changes;  pay 
for  employees  classified  in  this  band  (or  equivalent  range  of  grades) 
throughout  the  two-year  period  increased  by  similar  amounts  at 
Sacramento  and  the  comparison  sites.  (Among  all  continuing  em¬ 
ployees  who  started  in  the  DW2  band — whether  or  not  they  ended 
there — pay  grew  i  j  a  greater  extent  at  Sacramento.) 

The  overall  conclusion  is  that  there  was  evidence  of  greater  nominal 
salary  growth  at  Sacramento  during  the  first  two  years  of  the  demon¬ 
stration.  Average  pay  for  the  bands  containing  the  most  employees 
(DH2  and  DW2)  rose  significantly  more  at  Sacramento;  other  bands 
generally  changed  comparably,  and  two  instances  of  relative  declines 
appear  to  result  from  composition  changes.  The  fact  that  employees 
fared  as  well  or  better  financially,  especially  at  the  nonsenior,  non¬ 
supervisor  levels,  is  consistent  with  the  goals  of  PACER  SHARE, 
given  that  the  total  wage  bill  did  not  increase  (as  we  shall  see  later). 
Moreover,  there  was  little  evidence  of  a  pay  inversion  problem  be¬ 
tween  supervisory  and  nonsupervisory  positions.  For  the  most  part, 
Sacramento’s  supervisory  salaries  remained  higher  than  those  at  the 
comparison  ALCs  and  above  those  of  the  comparable  pay  grades  in 
the  white-collar  nonsupervisory  schedule  ^e.g.,  DX2  vs.  DW3). 
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Crossovers 

Table  21  examines  blue-  to  white-collar  and  white-  to  blue-collar 
crossovers  (i.e.,  job  changes)  among  nonsupervisory  personnel.  This 
corresponds  to  measure  II.A.2  in  the  evaluation  model.  (See  Table 
C.ll  in  N-3404-FMP.)  As  noted,  the  comparatively  high  wages  for 
blue-collar  employees  at  the  Sacramento  ALC  have  raised  concerns 
about  the  possibility  of  such  crossovers.  Analyses  based  on  the  data 
in  Table  20  suggest  that  the  blue-  versus  white-collar  gap  did  not 
close  appreciably  at  Sacramento  during  the  first  two  years  of  the 
demonstration.  Formally,  our  hypothesis  is  that  the  change  in 
crossover  rates  at  Sacramento  over  the  demonstration’s  first  two 
years  is  no  different  from  the  change  in  the  comparison  group. 
Rejection  of  the  hypothesis  due  to  a  relative  increase  at  Sacramento 
impli.  s  an  unfavorable  effect  of  the  demonstration;  a  decrease  would 
be  a  favorable  outcome. 

Baseline.  Concerns  over  crossovers  appear  to  be  exaggerated. 
Crossover  rates  ranged  from  near  0  to  just  over  1  percent.  The 
Sacramento  rates  did  not  differ  significantly  from  those  of  the  com¬ 
parison  group  for  crossovers  in  either  direction.  Given  the  very  low 
crossover  rates,  this  measure  will  focus  on  the  extent  to  which  the 
Sacramento  rates  remain  comparable  to  the  others.  It  is  not  reason¬ 
able  to  expect  a  comparative  decrease  in  crossovers  at  Sacramento 
unless  rates  rise  throughout  the  rest  of  the  system. 

Year  Two.  During  the  first  two  years  of  the  demonstration,  both 
types  of  crossovers  increased  in  the  comparison  group  by  about  one- 
half  of  one  percentage  point.  The  increase  for  white-  to  blue-collar 
crossovers  was  statistically  significant,  but  neither  change  was  large 
in  absolute  terms.  There  was  no  evidence  of  differential  change  at 
Sacramento  among  the  group  of  original  concern — white  to  blue  col¬ 
lar.  Crossovers  from  blue-  to  white-collar  jobs  did  increase  by  2.6  per¬ 
centage  points,  a  significant  difference  as  compared  with  the  .5  per¬ 
centage  point  increase  at  the  other  sites.  However,  this  difference  is 
largely  explained  by  a  single  reclassification  action  that  affected  12 
people  in  the  SPAALS  (Shipment  Planning  and  Address  Labeling 
System)  unit.  When  the  demonstration  began,  they  were  converted 
from  white  collar  to  blue.  They  petitioned  0PM  to  be  reclassified  as 
white  collar,  a  change  that  was  approved  and  accomplished  during 
the  reporting  period.  This  reclassification  accounted  for  1.7  of  the  2.6 
percentage  point  crossover  total. 
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Table  21 

Changes  Associated  with  Revised  Base  Pay 
Determination:  Crossovers 
(measure  11^.2  in  evaluation  model,  in  percent,  non¬ 
supervisors  only) 


Type  of 
Crossover 

Sacramento 

Baseline 

Rate 

Sacramento 
Baseline  DifT. 
from  Other  ALCs 

Year  2 
Change  for 
Other  ALCs 

Year  2 
Change  at 
Sacramento 

Blue  collar  to  white 
collar  (DH  to  DW) 

0.4 

-0.6 

0.5 

2.6* 

White  collar  to  blue 
collar  (DW  to  DH) 

0.3 

-0.3 

0.5‘ 

0.6 

NOTE:  Asterisk  in  column  two  or  three  indicates  the  coeiTlcient  is  significant 
at  the  p  <  .05  level;  in  column  four,  it  indicates  the  difference  in  the  coefTicients 
for  columns  three  and  four  is  significant  at  p  <  .05. 


VARIABLES  AFFECTED  BY  REVISED  SUPERVISORY 
GRADING  CRITERIA 

Tables  22  and  23  present  results  concerning  the  levels  of  supervision 
and  percentage  of  supervisory  personnel.  The  results  correspond  to 
measures  III.C.l  and  III.C.2  in  the  evaluation  model,  respectively. 
(See  Tables  C.3-C.6  in  N-3404-FMP.)  Recall  that  a  goal  of  the  re¬ 
vised  supervisory  grading  criteria  is  to  facilitate  a  streamlined  orga¬ 
nizational  structure  in  which  supervisors  can  be  assigned  to  the  posi¬ 
tions  where  they  are  required  without  regard  to  the  number  and 
grades  of  subordinates  supervised.  Because  of  the  decoupling  of  su¬ 
pervisory  positions  from  the  number  and  grades  of  subordinates  su¬ 
pervised,  we  would  anticipate  possible  changes  in  two  areas — in  the 
distribution  of  supervisors  by  level  of  supervision,  and  in  increased 
variation  in  the  percentages  of  supervisors  among  the  various  divi¬ 
sions  within  DS,  The  percentage  of  all  supervisors  in  the  total  work 
force  might  also  change;  the  direction  of  the  effect  is  difficult  to  antic¬ 
ipate,  however,  since  beneficial  gains  in  supervisory  percentage  in  one 
segment  of  the  organization  could  be  offset  by  beneficial  reductions  in 
other  segments.  Testable  hypotheses  relevant  to  the  measures  in 
Tables  22  and  23  are  thus  as  follows: 

•  Changes  in  the  distribution  of  supervisors  by  supervision  level  at 
Sacramento  are  the  same  over  the  first  two  years  of  the  demon¬ 
stration  as  those  in  the  comparison  group.  (Rejection  implies 
greater  flexibility.) 
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•  Changes  in  the  distribution  of  supervisors  by  division  at 
Sacramento  are  the  same  as  those  in  the  comparison  group. 
(Rejection  again  implies  flexibility.) 

Table  22  indicates  little  variation  in  the  percentage  of  supervisory 
personnel  across  the  ALCs  at  baseline.  The  total  percentage  of  super¬ 
visors  and  the  percentage  of  supervisors  at  each  level  did  not  differ 
significantly  at  baseline  between  Sacramento  and  the  comparison 
group.  In  addition,  there  were  no  significant  changes  during  the  first 
two  years  of  the  demonstration  for  the  ALCs  in  general  or 
Sacramento  in  particular.  Table  23  indicates  that  the  percentage  of 


Table  22 

Changes  Associated  with  Revised  Supervisory  Grading  Criteria: 
Supervisors  as  Percentage  of  Work  Force,  by  Level 
(measure  III.C.l  in  evaluation  model) 


Supervision 

Level 

Sacramento 

Baseline 

Rate 

Regression  Coefficient 

Sacramento 
Baseline  Diff. 
from  Other  ALCs 

Year  2 
Change  for 
Other  ALCs 

Year  2 
Change  at 
Sacramento 

1 

7,6 

-0.2 

-0,2 

-0.1 

2 

2.3 

-0.2 

0.1 

0.1 

3 

0.6 

-0.0 

-0.1 

0.1 

Total,  1,  2,  or  3 

10.4 

-0.4 

-0.2 

0.1 

Table  23 

Changes  Associated  with  Revised  Supervisory  Grading  Criteria: 
Percentage  of  Supervisors  by  Division 
(measure  I1I.C.2  in  evaluation  model) 


Regression  Coefficient 


Division 

Sacramento 

Baseline 

Rate 

Sacramento 
Baseline  Diff. 
from  Other  ALCs 

Year  2 
Change  for 
Other  ALCs 

Year  2 
Change  at 
Sacramento 

Materiel  Processing 
(DSF) 

8.9 

-O.l 

-0.2 

-0.9 

Management  Services 
(DSM) 

10.4 

-0.9 

0.2 

2.1 

Supply  (DSS) 

10.3 

-0.6 

-0.1 

1.7 

Transportation 
Operations  (DST) 

11.7 

-0.9 

-0.0 

0.7 
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supervisors  within  divisions  also  was  comparable  across  the  ALCs. 
At  baseline,  percentages  of  supervisors  by  division  at  the  Sacramento 
ALC  showed  no  statistical  differences  from  the  percentages  in  the 
same  divisions  for  the  comparison  group.  And,  there  were  no  signifi¬ 
cant  changes  during  the  first  two  years  of  PACER  SHARE.  Overall, 
then,  the  supervisor  results  provide  no  evidence  of  differential  change 
in  the  allocation  of  supervisors  at  Sacramento  during  the  first  two 
years  of  the  demonstration. 

VARIABLES  AFFECTED  BY  A  COMBINATION  OF 
INTERVENTIONS 

Percentage  of  Career  Employees 

There  are  at  least  three  reasons  for  assessing  possible  changes  in  the 
percentage  of  career  employees  during  this  period.  For  one,  the 
Demonstration  On-Call  program  will  provide  greater  flexibility  in  the 
release  and  recall  of  workers  and  in  their  conversion  to  career  status, 
depending  on  workload  demands.  Work  force  adjustments  will  be 
made  in  the  noncareer  Demonstration  On-Call  complement  whenever 
possible.  Thus,  a  decline  in  the  workload  should  not  necessitate  a  re¬ 
duction  in  the  career  force  to  the  same  extent  as  at  the  comparison 
sites.  Second,  to  the  extent  that  productivity  rises,  positions  vacated 
through  natural  attrition  may  not  be  refilled.  This  too  would  raise 
the  percentage  of  the  career  force  relative  to  the  other  ALCs.  Finally, 
the  opposite  trend  could  occur  if  negative  attitudes  toward  PACER 
SHARE  increased  attrition  among  senior  personnel.  Results  concern¬ 
ing  the  percentage  of  career  employees  at  each  of  the  ALCs  are 
presented  in  Table  24  by  pay  schedule.  The  results  correspond  to 
measure  VI.D.4  in  the  evaluation  model.  (See  Tables  C.7  and  C.8  in 
N-3404-FMP.)  Formally,  we  are  testing  the  hypothesis  that  the  per¬ 
centage  of  career  employees  did  not  change  differentially  at 
Sacramento  relative  to  the  comparison  group. 

Baseline.  The  percentage  of  career  employees  ranged  from  80  to  85 
percent  for  the  nonsupervisory  schedules  to  nearly  100  percent  for  the 
supervisory  schedule.  The  percentage  of  career  employees  at 
Sacramento,  relative  to  the  comparison  ALCs,  was  significantly  lower 
for  the  nonsupervisory  pay  schedules.  The  percentage  of  career  em¬ 
ployees  among  the  supervisors  did  not  differ  significantly. 

Year  Two.  As  can  be  seen  in  the  third  column  of  Table  24,  changes 
in  the  percentage  of  career  employees  at  the  other  ALCs  were  mixed. 
The  career  percentage  declined  significantly  among  blue-collar  non¬ 
supervisors,  remained  similar  among  white-collar  nonsupervisors. 
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Table  24 

Changes  Associated  with  a  Combination  of  Interventions: 
Percentage  of  Career  Employees  by  Pay  Schedule 
(measure  VI.D.4  in  evaluation  model) 


Regression  Coefficient 

Sacramento 

Sacramento 

Year  2 

Year  2 

Baseline 

Baseline  Diff. 

Change  for 

Change  at 

Pay  Schedule 

Rate 

from  Other  ALCs 

Other  ALCs 

Sacramento 

Blue-collar  nonsuper¬ 
visors  (DH) 
White-collar  non- 

77.8 

-3.3* 

-2.9* 

5.0* 

supervisors  (DW) 

81.6 

-4.8* 

0.7 

6.4* 

Supervisors  (DX) 

97.3 

-0.5 

1.6* 

-0.3 

NOTE:  Asterisk  in  column  two  or  three  indicates  the  coefficient  is  significant  at 
the  p  <  .05  level;  in  column  four,  it  indicates  the  difference  in  the  coefficients  for 
columns  three  and  four  is  significant  at  p  <  .05. 


and  increased  among  supervisors.  Whereas  the  career  percentage 
among  nonsupervisors  was  level  or  declined  elsewhere,  this  was  not 
true  at  Sacramento.  The  percentage  of  career  employees  rose  for  the 
DH  and  DW  pay  schedules,  and  the  difference  between  Sacramento 
and  the  comparison  sites  was  statistically  significant.  There  was  no 
change  in  the  percentage  of  career  employees  among  supervisors  at 
Sacramento  relative  to  the  comparison  sites.  On  balance,  then,  the 
results  reflect  the  fact  that  senior  staff  did  not  disproportionately 
leave  DS  at  Sacramento  as  a  result  of  PACER  SHARE  and  that  non¬ 
career  positions  vacated  through  natural  attrition  were  not  filled  to 
the  extent  they  were  at  the  other  ALCs.  (The  former  point  will  be¬ 
come  clearer  in  our  discussion  of  turnover,  below.)  New  hires  were 
substantially  lower  at  Sacramento  than  at  the  comparison  sites  dur¬ 
ing  the  demonstration  period,  according  to  official  personnel  records. 
For  both  blue-  and  white-collar  nonsupervisors,  we  thus  reject  the 
null  hypothesis  of  no  differential  change  in  the  percentage  of  the  work 
force  composed  of  career  personnel. 


Turnover 

The  next  four  tables  present  turnover  information.  Instances  of 
turnover  are  distinguished  according  to  whether  they  reflect  separa¬ 
tions  or  internal  transfers  (migrations)  to  other  directorates  at  the 
given  base.  The  results  correspond  to  measure  VI.D.l  in  the  evalua¬ 
tion  model.  (See  Tables  C.12-C.15  in  N-3404-FMP.)  To  the  extent 
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that  PACER  SHARE  achieves  its  goal  of  improving  the  quality  of 
work  life,  we  would  expect  turnover  to  decrease  over  time  during  the 
demonstration.  Such  a  decrease,  however,  could  be  preceded  by  an 
initial  period  of  greater  turnovei,  during  w-hirh  time  staff  members 
adverse  to  PACER  SHARE  leave  DS.  Formally,  we  are  testing  the 
hypothesis  that  the  change  in  the  turnover  rate  at  Sacramento  in  the 
first  two  years  of  the  demonstration  is  the  same  as  that  in  the  com¬ 
parison  group.  Rejection  due  to  a  relative  decrease  at  Sacramento 
implies  a  beneficial  effect;  rejection  due  to  an  increase  implies  a 
harmful  effect.  The  results  in  the  four  tables  establish  the  turnover 
levels  for  DS  as  a  whole  and  for  specific  subgroups  of  th..  work  force 
for  which  changes  in  turnover  are  being  assessed.  Our  null  hypothe¬ 
ses  for  the  subgroups  are  analogous  to  that  for  DS  as  a  whole. 
However,  we  do  not  associate  rejection  with  a  beneficial  or  negative 
effect  beyond  that  established  for  DS  because  subgroups  showing  les.® 
turnover  than  DS  as  a  whole  must  be  balanced  by  subgroups  showing 
greater  turnover  than  DS  as  a  whole.  It  is  not  a  demonstration  objec¬ 
tive  that  any  subgroup  show  an  improvement  in  turnover  at  the  ex¬ 
pense  of  another  subgroup.  The  subgroup-by-subgroup  distribution  of 
the  overall  change  within  DS  is  simply  a  point  of  information. 

Table  25  shows  the  overall  DS  separation  and  internal  transfer  (mi¬ 
gration)  rates  during  baseline  (calendar  year  1987)  and  during  year 
two  of  PACER  SHARE.  The  total  turnover  figures  represent  the  sum 
of  the  separation  and  migration  rates.  Separations  are  defined  as  all 
who  left  their  installation  during  the  prior  year  (1987  for  the  baseline. 

Table  25 

Changes  Associated  with  a  Combination 
of  Interventions:  Turnover 
(measure  VI.D.l  in  evaluation  model;  in  percent) 


Type  of  Turnover 

Sacramento 

Baseline 

Rate 

Sacramento 
Baseline  DifT. 
from  Other  ALCs 

Year  2 
Change  for 
Other  ALCs 

Year  2 
Change  at 
Sacramento 

Separations 

Internal  tr^  •’=fers 

10.8 

2.3* 

-3.0* 

-3.7 

(migration) 

3.8 

1.0* 

0.8* 

-1.1* 

Total 

14.7 

3.3* 

-2.2* 

-4.7* 

NOTE:  Asterisk  in  column  two  or  three  indicates  the  coefficient  is  significant  at  the 
p  <  .05  level,  in  column  four,  it  indicates  the  difference  in  the  coefficients  for  columns 
three  and  four  is  significant  at  p  <  ,05. 
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April  1989  to  March  1990  for  year  two).^  The  denominator  in  the  rate 
calculation  is  the  total  DS  work  force  at  Sacramento  or  in  the 
comparison  group  at  the  end  of  the  last  month  prior  to  the  baseline 
year  or  the  second  year  of  the  demonstration. 

Baseline.  Overall,  the  baseline  turnover  rate  was  nearly  15  percent 
at  Sacramento.  Turnover  at  Sacramento  was  significantly  higher 
than  for  the  comparison  group.  Analysis  of  the  components  indicates 
that  separations  were  nearly  11  percent  at  Sacramento.  The 
Sacramento  rate  was  significantly  higher  than  the  average  separation 
rate  across  the  other  ALCs.  Internal  transfer  (migration)  to  other  di¬ 
rectorates  at  Sacramento  was  just  under  4  percent,  which  also  was 
significantly  higher  than  for  the  comparison  ALCs. 

Year  Two.  Overall,  turnover  decreased  by  2.2  percentage  points  at 
the  comparison  ALCs  during  year  two  of  PACER  SHARE.  The  drop 
in  total  turnover  reflected  an  offsetting  decrease  in  separation  (-3.0 
percentage  points)  and  increase  in  migration  (0.8  percentage  points). 
These  changes  were  all  statistically  significant.  As  compared  with 
the  other  ALCs,  internal  transfers  and  total  turnover  decreased  sig¬ 
nificantly  at  Sacramento.  (Separations  dropped  as  well,  although  the 
decrease  did  not  differ  significantly  from  that  at  the  other  ALCs.®)  As 
a  result,  the  year-two  turnover  rates  were  similar  for  Sacramento  and 
the  comparison  ALCs.  Because  Sacramento  had  greater  turnover  at 
baseline,  the  change  is  consistent  with  a  hypothesis  of  improved  qual¬ 
ity  of  work  life.  To  the  extent  turnover  actually  becomes  lower  at 
Sacramento,  the  evidence  will  become  even  more  persuasive. 

Table  26  shows  the  separation  and  internal  transfer  rates  by  career 
category. 

Baseline.  Among  career  employees,  the  separation  rate  at  Sacra¬ 
mento  (11.4  percent)  was  significantly  greater  than  at  the  other 
ALCs.  The  separation  rate  among  career-conditional  employees^  did 


^Separations  include  retirements  and  deaths.  We  believe  there  is  some  merit  to 
including  retirements  because,  like  resignations,  they  represent  departure  decisions 
that  are  at  least  partly  under  the  control  of  the  employee.  Deaths  in  the  work  force  are 
likely  to  be  negligible. 

^It  could  be  argued  that  lower  internal  transfer  rates  are  not  indicative  of  improved 
quality  of  work  life  if  they  result  from  inability  of  employees  to  transfer  because  of 
institutional  barriers  posed  by  PACER  SHARE. 

Career-conditional  employees  are  full-time  employees  who  have  not  yet  met  the 
three-year  criterion  for  career  employment  privileges;  at  Sacramento,  the  category  in¬ 
cludes  DOC  employees  after  baseline  (who  have  a  one-year  criterion). 
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Table  26 

Changes  Associated  with  a  Combination  of  Interventions:  > 

Turnover  by  Career  Category 

(measure  VI.D.l  in  evaluation  model;  percentage  of  career  category) 


Type  of  Turnover  and 
Career  Category 

Sacramento 

Baseline 

Rate 

Sacramento 
Baseline  Diff. 
from  Other  ALCs 

Year  2 
Change  for 
Other  ALCs 

Year  2 
Change  at 
Sacramento 

SEPARATIONS 

Career 

11.4 

2.5* 

-3.2* 

-5.4* 

Career-conditional 

8.0 

1.3 

-1.7 

12.0* 

INTERNAL  TRANSFERS  (MIGRATION) 

Career 

2.2 

0.3 

0.9* 

0.3 

Career-conditional 

12.2 

3.7* 

-1.4 

-6.0 

NOTE:  Asterisk  in  column  two  or  three  indicates  the  coefficient  is  significant  at 
the  p  <  .05  level;  in  column  four,  it  indicates  the  difference  in  the  coefficients  for 
columns  three  and  four  is  significant  at  p  <  .05. 


not  differ  statistically  between  Sacramento  and  the  comparison 
group.  The  internal  transfer  (migration)  rate  to  other  directorates  at 
Sacramento  was  not  significantly  higher  than  for  the  comparison 
ALCs  among  career  employees;  however,  it  was  significantly  higher 
among  career-conditional  employees.  When  considered  together  with 
the  separation  results,  the  data  indicate  that,  at  baseline,  most 
turnover  among  more  senior  (career)  employees  represented  separa¬ 
tions  whereas  among  more  junior  (conditional)  employees  it  repre¬ 
sented  transfers. 

Year  Two.  The  second-year  changes  for  the  comparison  group  in 
Table  26  indicate  a  significant  decline  in  separations  by  career  em¬ 
ployees  and  a  near-significant  decline  among  career-conditional  em¬ 
ployees.  Meanwhile,  transfers  went  in  opposite  directions  for  the  two 
groups — up  among  career  employees  and  down  among  the  career-con¬ 
ditional.  The  change  for  career  employees  was  statistically  signifi¬ 
cant,  although  it  was  not  large  in  absolute  terms  (0.9  percentage 
points).  Transfers  remained  higher  among  career-conditional  work¬ 
ers.  Unlike  baseline,  during  year  two  the  separation  rate  among  ca¬ 
reer  and  conditional  employees  was  similar  (11.4  -  2.5  -  3.2  =  5.7;  8.0 
-1.3 -1.7  =  5.0). 

Migration  decreased  at  Sacramento  within  the  two  career  categories 
relative  to  the  comparison  group;  however,  the  decreases  for  the  two 
individual  categories  were  not  statistically  significant.  (As  noted,  the 
total  decrease  for  the  two  categories  combined  was  significant.) 
Separations  reveal  a  different  and  noteworthy  pattern.  Among  career 
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employees,  separations  declined  significantly  at  Sacramento  relative 
to  the  comparison  sites.  Among  career-conditional  employees — DOCs 
and  first-year  employees  at  the  start  of  the  demonstration — ^the  sepa¬ 
ration  rate  increased  significantly  at  Sacramento.  If  this  were  linked 
to  the  intentional  separation  of  DOCs  or  of  employees  who  could  not 
be  converted  to  career  status  because  of  reduced  workload,  it  would 
support  the  goal  of  increased  flexibility  in  protecting  the  career  force 
when  making  separations  to  accommodate  reductions  in  workload. 
That  link,  however,  is  not  yet  established. 

Table  27  shows  turnover  information  by  pay  schedule. 

Baseline.  Overall,  the  results  suggest  that  differences  in  turnover 
behavior  between  Sacramento  and  the  comparison  ALCs  at  baseline 
tended  to  occur  among  white-collar  nonsupervisory  personnel  (who 
were  paid  less  well  relative  to  blue-collar  nonsupervisors  than  at 
other  ALCs,  probably  because  of  the  effects  of  local  wage  surveys  on 
blue-collar  wages).  The  separation  results  indicate  that  such  employ¬ 
ees  at  Sacramento  had  a  significantly  higher  separation  rate.  Among 
blue-collar  nonsupervisory  employees  and  among  supervisors,  the 
rates  were  statistically  equivalent.  Sacramento’s  internal  transfer 

Table  27 

Changes  Associated  with  a  Combination  of  Interventions: 

Turnover  by  Pay  Schedule 

(measvu'e  VI.D.1  in  evaluation  model;  percentage  of  pay  schedule) 


Sacramento 

Sacramento 

Year  2 

Year  2 

Type  of  Turnover  and 

Baseline 

Baseline  Diff. 

Change  for 

Change  at 

Pay  Schedule 

Rate 

from  Other  ALCs 

Other  ALCs 

Sacramento 

SEPARATIONS 

Blue-collar  nonsuper¬ 
visors  (DH) 
White-collar  non- 

9.4 

0.5 

1 

* 

-3.2 

supervisors  (DW) 

11.2 

3.5* 

-2.6* 

-2.5 

Supervisors  (DX) 

14.6 

3.5 

-6.5* 

-9.6 

INTERNAL  TRANSFERS  (MIGRATION) 

Blue-collar  nonsuper¬ 
visors  (DH) 
White-collar  non- 

1.8 

0.5 

2.1* 

0.2* 

supervisors  (DW) 

6.3 

1.7* 

-0.4 

-2.1 

Supervisors  (DX) 

1.4 

-0.2 

-0.1 

-0.8 

NOTE:  Asterisk  in  column  two  or  three  indicates  the  coeiTicient  is  significant  at 
the  p  <  .06  level;  in  column  four,  it  indicates  the  difference  in  the  coefficients  for 
columns  three  and  four  is  significant  at  p  <  .05. 
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rate  also  was  higher  than  the  comparison  group’s  among  white-collar 
nonsupervisory  employees.  For  the  other  two  pay  schedules,  transfer 
differences  were  not  statistically  significant. 

Year  Two.  As  can  be  seen  in  column  three,  the  overall  decrease  in 
separations  at  the  comparison  sites  during  year  two  held  true  for  all 
pay  schedules.  The  increase  in  migration,  however,  occurred  only 
among  blue-collar  nonsupervisors.  As  seen  in  column  four,  Sacra¬ 
mento  separations  showed  declines  within  each  pay  schedule  similar 
to  those  occurring  elsewhere.  The  change  in  internal  transfer  rate 
also  was  comparable  among  supervisors  and  white-collar  non- 
supervisors  across  sites.  Transfers,  however,  were  significantly  less 
common  at  Sacramento  than  elsewhere  among  blue-collar  nonsuper¬ 
visors;  they  did  not  reflect  the  increase  seen  at  the  comparison  sites. 

Finally,  Table  28  shows  turnover  information  by  division. 

Table  28 

Changes  Associated  with  a  Combination  of  Interventions: 

Turnover  by  Division 

(measure  V1.D.1  in  evaluation  model;  percentage  of  division) 


Type  of  Turnover 
and  Division 

Sacramento 

Baseline 

Rate 

Sacramento 
Baseline  Diff. 
from  Other  ALCs 

Year  2 
Change  for 
Other  ALCs 

Year  2 
Change  at 
Sacramento 

SEPARATIONS 

Materiel  Processing 
(DSF) 

9.3 

0.9 

-2.9* 

-2.3 

Management  Services 
(DSM) 

12.9 

4.8* 

-3.1* 

-3.1 

Quality  Management 
(DSQ) 

18.3 

7.6 

-5.4* 

-13.9 

Supply  (DSS) 

11.3 

3.4* 

-2.8* 

^.5 

Transportation 
Operations  (DST) 

10.8 

1.6 

-2.7* 

-3.2 

INTERNAL  TRANSFERS  (MIGRATION) 

Materiel  Processing 

(DSF)  3.4  1.1 

1.8* 

-1.5* 

Management  Services 
(DSM) 

6.4 

2.7 

0.8 

-3.4 

Quality  Management 
(DSQ) 

0.0 

-2.7 

0.1 

0.0 

Supply  (DSS) 

5.0 

1.2 

-0.7 

0.3 

Transportation 
Operations  (DST) 

3.0 

0.6 

0.5 

-0.6 

NOTE:  Asterisk  in  column  two  or  three  indicates  the  coefficient  is  significant  at  the 
p  <  .05  level;  in  column  four,  it  indicates  the  difference  in  the  coefficients  for  columns 
three  and  four  is  significant  at  p  <  .06. 
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Baseline.  The  data  show  some  variation  in  turnover  behavior. 
Separations  for  the  Division  of  Management  Services,  Quality  Man¬ 
agement,  and  Supply  were  significantly  or  marginally  higher  at 
Sacramento.  The  Materiel  Processing  and  Transportation  Operations 
Division  separation  rates  were  similar  for  Sacramento  and  the  com¬ 
parison  ALCs.  The  pattern  of  division  transfer  (migration)  rates  was 
more  comparable  across  the  ALCs.  Only  Materiel  Processing  and 
Management  Services  showed  marginally  more  transfers  at 
Sacramento.  None  of  the  other  differences  in  Sacramento  rates  at 
baseline  approached  statistical  significance. 

Year  Two.  The  results  for  year  two  indicate  that  the  decrease  in 
separations  at  the  comparison  sites  was  significant  for  all  divisions, 
whereas  migrations  increased  significantly  only  for  DSF.  The  overall 
Sacramento  pattern  of  separations  relative  to  the  other  ALCs  held 
true  at  the  division  level:  the  changes  in  separation  rates  were  statis¬ 
tically  equivalent  to  those  at  the  comparison  sites  for  each  division. 
Migrations  declined  significantly  at  Sacramento  for  DSF  and 
marginally  for  DSM  relative  to  the  changes  for  those  divisions  at  the 
comparison  sites.  Changes  in  the  migration  rate  were  similar  for  the 
remaining  divisions. 

SUMMARY 

We  now  summarize  results  from  the  automated  personnel  system. 
Our  analysis  indicates  general  improvement  from  baseline  levels  at 
Sacramento  relative  to  changes  at  the  other  ALCs  during  this  time 
period.  Consistent  with  both  increased  organizational  flexibility 
(greater  salary  potential  through  pay  banding  and  annual  increases) 
and  quality  of  work  life  (higher  pay),  average  salary  growth  in  DS 
during  the  two-year  period  was  greater  at  Sacramento  than  at  the 
other  ALCs.  The  change  varied  by  pay  band,  but  wages  in  the  largest 
bands  increased  significantly  relative  to  the  comparison  sites.  Among 
continuing  employees,  salaries  grew  at  a  rate  equal  to  or  greater  than 
those  for  comparable  pay  bands  at  the  other  ALCs.  There  was  no  evi¬ 
dence  of  pay  inversion  between  supervisors  and  nonsupervisors. 
These  results  are  noteworthy  in  showing  that  most  employees  fared 
as  well  or  better  financially  under  job  series  consolidation  and  pay 
banding  than  did  their  counterparts  at  the  other  ALCs — operating 
under  the  traditional  system — despite  the  worsening  of  pay-related 
attitudes.  In  other  words,  pay  dissatisfaction  appears  to  be  a  matter 
of  perception.  Supplemental  data  suggest  that  the  cost  of  living  rose 
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more  rapidly  in  the  Sacramento  area  than  at  the  comparison  sites.® 
During  the  demonstration  period,  Sacramento’s  cost  of  living  rose  ap¬ 
proximately  10  percent  more  than  the  national  average;  at  the  com¬ 
parison  sites,  the  cost  of  living  increased  by  approximately  5  percent 
less  than  the  national  average.  This  difference  may  have  contributed 
to  the  worsening  of  pay  satisfaction.  The  worsening  of  attitudes  con¬ 
cerning  the  link  between  job  performance  and  compensation  or  ad¬ 
vancement,  however,  is  more  likely  tied  to  the  changes  in  annual 
salary  increases  and  promotion  practices  under  job  series  consolida¬ 
tion  and  pay  banding.  It  also  should  be  noted  that  the  salary  growth 
was  offset  by  a  reduction  in  the  size  of  the  work  force.  As  discussed 
below,  overall  labor  costs  did  not  increase. 

Two  other  measures  of  organizational  flexibility  concerned  the  num¬ 
ber  of  skills  per  employee  and  the  percentage  of  supervisors. 
Consistent  with  the  goals  of  expanding  training  and  increasing  orga¬ 
nizational  flexibility,  the  average  number  of  skills  grew  significantly 
more  at  Sacramento  during  the  two-year  period;  that  is,  more  skill 
training  appears  to  have  been  provided,  amounting  to  about  one  more 
skill  for  every  three  workers  than  at  the  other  ALCs.  In  contrast,  the 
percentage  of  supervisors  remained  unchanged,  both  overall  and 
within  divisions.  Although  this  suggests  that  the  flexibility  provided 
by  changes  in  supervisory  grading  criteria  was  not  used  to  proliferate 
supervisory  positions,  it  also  suggests  that  the  flexibility  has  not  yet 
been  fully  utilized.  (Movement  of  supervisors  within  divisions  could 
have  occurred.) 

Finally,  two  measures  of  turnover  were  examined  as  they  bear  on  the 
quality  of  work  life  and  on  hiring  and  separation  practices.  It  was 
believed  that  the  percentage  of  career  employees  might  decrease  if 
senior  personnel  were  unhappy  with  PACER  SHARE.  In  fact,  the 
results  show  that  the  percentage  of  career  employees  grew 
significantly  at  Sacramento  relative  to  that  at  the  comparison  sites 
during  the  two-year  period.  In  addition  to  its  consistency  with 
possible  improvement  in  the  quality  of  work  life,  this  outcome  reflects 
fewer  hiring  actions  taken  at  Sacramento  to  fill  vacated  positions,  and 
it  is  consistent  with  the  goal  of  retaining  key  personnel  through  the 
DOC  program.  Also  consistent  with  greater  quality  of  work  life,  the 
turnover  rate  decreased  significantly  at  Sacramento.  It  thus  became 
similar  to  that  at  the  other  ALCs,  rather  than  maintaining  its 
historically  higher  rate. 


^Composite  Cost  of  Living  Index,  American  Chamber  of  Commerce  Research 
Association,  quarterly  reports,  1985-1990. 


6.  RESULTS  FOR  WORK  QUALITY  MEASURES 


We  now  consider  measures  of  work  quality  (VI.B.l  in  the  evaluation 
model).  As  we  indicated  earlier,  one  of  the  goals  of  PACER  SHARE 
was  to  maintain  work  quality  while  increasing  productivity  and,  in 
the  longer  term,  to  improve  quality  as  well.  The  quality  measures 
evaluated  fall  into  two  groups:  (1)  measures  of  error  rates  for  particu¬ 
lar  processes  such  as  packing  (Quality  Management  Division  or  DSQ 
measures)  or  as  assessed  in  reports  of  discrepancies  (RODs)  in 
shipped  (or  received)  items  and  (2)  measures  of  success  in  meeting 
timeliness  and  support  goals  (Management  Services  Division  or 
DSMPA  measures).  During  the  evaluation  period,  collection  of  data 
for  reports  of  discrepancies  and  timeliness/support  measures  was  di¬ 
rected  by  the  Air  Force  Logistics  Command  (AFLC)  and  was  per¬ 
formed  consistently  across  the  ALCs.  Therefore,  these  measures  af¬ 
ford  a  standard  of  comparison  across  the  Air  Logistics  Centers  and 
have  been  chosen  to  constitute  the  measures  used  in  our  analysis. 
This  also  applied  to  the  DSQ  error  rate  measures  during  the  baseline 
and  first-year  periods.  During  year  two,  AFLC  discontinued  the  di¬ 
rective  to  collect  these  data.  As  a  result,  we  will  restrict  our  analysis 
of  error  rate  changes  to  Sacramento,  where  the  measures  continue  to 
be  collected  under  the  prior  procedures.^  Monthly  data  observations 
for  the  ALCs  covering  1985-1987  were  combined  to  form  the  baseline 
period,  and  each  month  was  weighted  equally Each  of  the  quality 
categories  will  be  discussed  in  turn.  Full  annual  site-specific  results, 
regression  analyses,  and  statistics  for  these  measures  are  shown  in 
App.  D  of  N-3404-FMP. 

ERROR  RATES  AND  REPORTS  OF  DISCREPANCIES 

Table  29  presents  the  results  for  the  DSQ  and  ROD  measures. 
Measures  are  distinguished  by  the  area  of  work  they  pertain  to  and 
the  particular  error  or  problem  rate  they  involve. 


Additional  measures  can  be  maintained  through  local  initiatives.  In  this  manner, 
six  of  eight  error  measures  previously  included  in  our  analysis  were  retained  by 
Sacramento;  monthly  data  collection  for  two  storage  measures  was  discontinued.  New 
measures  will  be  considered  for  inclusion  in  the  evaluation  in  the  future  to  the  extent 
they  become  directed  by  AFLC  or  are  sufficiently  complete  to  afford  meaningful  com¬ 
parisons. 

2See  R-3763-FMP  for  discussion  of  the  analysis  underlying  this  procedure. 
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We  begin  by  looking  at  results  of  ordinary  least-squares  regression 
analyses  performed  for  the  DSQ  and  ROD  measures  of  (monthly)  er¬ 
ror  rates.  The  regression  model  employed  was  similar  to  that  used  in 
the  multiple  skill  analyses  reported  in  Table  19,  except  that,  as  noted, 
the  DSQ  measure  analyses  used  year-two  data  for  Sacramento  alone 
(and  t-tests  were  used  to  assess  the  significance  of  the  two-year 
change  at  Sacramento  relative  to  its  baseline;  see  App.  D  of  N-3404- 
FMP).  If  we  read  across  the  first  row,  we  can  see  that  the  error  rate 
prior  to  the  demonstration  on  controlled  exceptions  (or  BL7)  was  7.3 
percent.  In  this  case,  the  regression  coefficient  shown  in  the  next  col¬ 
umn  indicates  that  the  error  rate  was  significantly  higher  than  it  was 
at  the  other  ALCs  by  5.2  percentage  points;  in  other  words,  it  was  2.1 
percent  at  the  other  ALCs.  The  change  at  S-  ''ento  during  year 
two  is  shown  in  the  last  column.  Accordi;  .  .ne  results,  the  error 
rate  decreased  significantly  by  4  percentage  points. 

If  we  look  at  the  broad  pattern  of  results,  we  note  that  with  the  one 
exception  of  the  measure  just  discussed,  error  rates  generally  were  in 
the  area  of  1  percent  or  less  at  baseline  at  Sacramento.  As  can  be 
seen  in  the  second  column,  the  error  rates  at  Sacramento  tended  to  be 
lower  than  they  were  at  the  other  ALCs,  that  is,  there  was  better 
quality  to  begin  with:  on  four  of  the  six  measures,  the  rate  was  signif¬ 
icantly  lower.  Finally,  column  four  indicates  that,  overall,  year-two 
error  rates  at  Sacramento  tended  to  be  comparable  to  those  at  base¬ 
line.  Three  measures  showed  significant  improvement  (BL7,  RL2, 
and  VLl),  two  showed  significant  increases  (PL4  and  VL3),  and  one 
showed  no  change  (RL5).  In  short,  then,  work  quality  as  assessed  by 
the  DSQ  measures  began  at  a  superior  level  at  Sacramento  and  the 
error  rate  across  all  measures  remained  about  the  same  there 
through  year  two  of  the  demonstration. 

The  lower  portion  of  the  table  shows  RODs.  Reports  of  discrepancies 
indicate  problems  or  inconsistencies  with  shipments  from  DS  (RODs 
received)  or,  secondarily,  with  shipments  into  DS  (RODs  initiated). 
The  ROD  results  show  much  the  same  pattern  as  for  the  DSQ  mea¬ 
sures.  Error  rates  were  generally  1  percent  or  less  and  were  lower  at 
Sacramento  than  at  the  other  ALCs  prior  to  the  demonstration. 
During  year  two,  there  was  no  change  at  the  other  ALCs  on  these 
measures,  and  there  was  no  differential  change  between  Sacramento 
and  the  other  sites. ^ 


3ln  the  case  of  RODs  initiated,  it  can  be  argued  that  a  higher  rate  results  from 
greater  scrutiny  of  receipts  by  DS,  rather  than  poorer  quality  of  receipts  (a  situation 
that  one  would  like  to  see  corrected  over  time).  In  this  context,  it  may  be  noted  that  the 
difference  in  the  rate  of  RODs  initiated  by  Sacramento  and  the  comparison  ALCs  is 
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TIMELINESS  AND  SUPPORT 

In  Table  30  we  examine  the  Management  Services  Division  measures 
of  compliance  with  receiving  tirtieliness  and  issue  support  goals. 
Receiving  documents  posted  within  standard  (one  day)  and  receipts 
binned  within  standard  (two  days)  are  both  measures  of  receiving 
timeliness,  with  the  optimum  performance  being  100  percent.  The 
official  goals  are  90  and  70  percent,  respectively.  For  the  third  and 
fourth  measures,  which  pertain  to  shipping  support,  the  lower  the 
rate,  the  better. 

The  first  column  indicates  the  mean  timeliness/support  rate  (in  per¬ 
cent)  during  the  baseline  period  for  each  measure  at  Sacramento. 
The  baseline  period  extends  from  the  date  of  publication  of  the  cur¬ 
rent  standard  through  December  1987.  The  period  varies  somewhat 
across  the  measures.  The  start  dates  are  September  1985  for  the  first 
two  measures,  October  1985  for  high-priority  requisitions,  and 
January  1985  for  the  denial  rate  measure.  As  before,  the  statistical 

Table  30 

Results  for  Measures  of  Work  Quality:  Management 
Division  Indicators 

(measure  Vl.B.l  in  evaluation  model;  percentage  of  receipts/issues) 


Regression  Coefficient 


Management 
Services  (DSMPA) 
Measure 

Sacramento 

Baseline 

Rate 

Sacramento 
Baseline  Diff. 
from  Other  ALCs 

Year  2 
Change  for 
Other  ALCs 

Year  2 
Change  at 
Sacramento 

Receiving  documents 
posted  within  one  day 
(goal  =  90%) 

97.6 

4.9* 

4.4* 

0.2* 

Receipts  binned  within 
two  days  (goal  =  70%) 

82.7 

9.4* 

-0.8 

-26.7* 

High-priority  requisitions 
(ceiling  =  30%) 

28.7 

-2.8 

-1.5 

5.9* 

Denial  rate  (ceiling  =  1%) 

1.0 

0.2* 

-0.1 

0.2* 

NOTE:  Asterisk  in  column  two  or  three  indicates  the  coefficient  is  significant  at  the 
p  <  .05  level;  in  column  four,  it  indicates  the  difference  in  the  coefficients  for  columns 
three  and  four  is  significant  at  p  <  .05. 


largely  attributable  to  the  initiation  rate  at  Ogden,  which  is  higher  than  the  others.  If 
we  remove  the  Ogden  data,  the  Sacramento  results  are  comparable  with  those  for  the 
remaining  ALCs. 
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significance  of  the  Sacramento  baseline  rate  as  compared  with  the 
rate  for  the  comparison  group  and  changes  during  year  two  were 
evaluated  using  ordinary  least-squares  regression. 

Overall,  predemonstration  timeliness  indicators  generally  met  appli¬ 
cable  standards,  and  Sacramento  ALC  timeliness/support  tended  to 
be  as  good  aS  or  better  than  that  at  the  other  ALCs.  The  percentage 
of  receiving  documents  posted  and  receipts  binned  within  standard 
was  significantly  greater  at  Sacramento,  and  the  percentage  of  high- 
priority  requisitions  was  nonsignificantly  lower.  Denial  rate  mea¬ 
sures  the  inability  to  fill  a  request  for  an  item  that  was  believed  to  be 
available.  The  Sacramento  denial  rate  was  significantly  higher,  al¬ 
though  the  difference  was  not  large  in  absolute  terms. 

During  year  two,  the  percentage  of  receiving  documents  posted  within 
standard  improved  at  the  other  ALCs;  there  was  no  change  in  binning 
timeliness.  On  both  these  measures,  however,  timeliness  decreased 
at  Sacramento  relative  to  the  other  ALCs.  In  the  case  of  posting  re¬ 
ceiving  documents,  the  timeliness  rate  at  the  other  ALCs  simply  be¬ 
came  more  similar  to  Sacramento’s,  which  already  was  close  to  100 
percent.  However,  there  was  a  binning  decrease  at  Sacramento. 
Some  portion  of  the  decrease  in  binning  timeliness  may  be  at¬ 
tributable  to  factors  associated  with  the  implementation  of  the 
Automated  Warehouse  System  (AWS).  The  system’s  implementation 
schedule  at  Sacramento  varies  from  that  at  other  ALCs.  For  exam¬ 
ple,  initially,  AWS  was  implemented  at  Ogden,  and  timeliness  did  de¬ 
cline  there  in  1987  during  phase-in,  although  that  has  not  been 
definitively  linked  to  implementation.  Moreover,  management  em¬ 
phasis  on  binning  could  change,  because  the  AWS  makes  it  possible  to 
locate  an  item  for  shipment  even  if  it  is  not  yet  binned.  Additional 
data  are  required  to  examine  the  linkage  at  Sacramento  and  to  dis¬ 
tinguish  unique  factors  from  those  common  to  other  ALCs. 

The  third  measure  concerns  high-priority  requisitions.  During  year 
two,  there  was  no  significant  change  in  that  rate  at  the  other  ALCs. 
The  rate  rose  by  about  6  percentage  points  at  Sacramento,  however, 
representing  a  significant  increase  relative  to  the  other  sites. 
Sacramento  believes  that  support  for  the  F-15  program  may  account 
for  the  difference.  Finally,  during  year  two,  there  was  no  significant 
change  in  the  denial  rate  at  the  other  ALCs,  which  declined  by  .1  per¬ 
centage  points.  At  Sacramento,  it  increased  by  .2  points.  The  relative 
difference  (.3  percentage  points)  was  statistically  significant,  although 
it  is  not  large  in  absolute  terms. 
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SUMMARY 

Our  analysis  of  work  quality  focused  on  command-directed  measures 
of  error  rates  and  RODs  maintained  by  the  Quality  Division  and  mea¬ 
sures  of  timeliness  and  support  maintained  by  the  Management 
Division.  The  error  rates  for  controlled  exceptions,  packing  process, 
receiving  inspection,  tailgate  accuracy,  location  audit  program  survey, 
and  physical  count  of  noncontrolled  items  showed  no  overall  pattern 
of  change  from  baseline,  at  which  time  rates  at  Sacramento  were  su¬ 
perior  to  those  at  the  comparison  sites.  Similarly,  reports  of  discrep¬ 
ancies  showed  no  overall  pattern  of  change  from  baseline,  remaining 
lower  at  Sacramento.  Finally,  there  was  a  relative  decline  in  timeli¬ 
ness  and  support — as  assessed  by  receiving  document  posting,  bin¬ 
ning,  high-priority  requisitions,  and  denials — relative  to  the  other 
ALCs.  The  two  largest  changes  occurred  for  binning  timeliness  and 
requisitions  and  apparently  are  at  least  partially  attributable  to  AWS 
implementation  and  management  decisions  such  as  support  of  the 
F-15. 


6.  ANAL  YSIS  OF  COST  SAVINGS 


We  seek  to  learn  whether  PACER  SHARE  led  Sacramento  to  achieve 
cost  savings  relative  to  what  would  have  been  expected  from  its  pre- 
PACER  SHARE  performance,  and  more  tellingly,  whether  PACER 
SHARE  produced  cost  savings  when  judged  in  comparison  with  other 
ALCs’  performance.  Some  turbulence  occurs  in  any  field  demonstra¬ 
tion,  and  that  has  been  the  case  with  PACER  SHARE.  Sacramento 
coped  with  a  temporary  hiring  freeze  in  summer  1988,  followed  by  a 
freeze  beginning  January  1990  and  still  in  effect.  Along  the  way,  the 
Directorate  faced  prospects  of  base  closure,  absorption  by  the  Defense 
Logistics  Agency,  and  on-base  recrganization.  Still,  there  are  com¬ 
mon  threads  across  the  ALCs.  Other  ALCs  in  addition  to  Sacramento 
felt  the  hiring  freezes  and  contemplated  absorption  by  the  Defense 
Logistics  Agency.  Although  less  pressing  at  other  ALCs,  base  closure 
loomed  as  a  possibility,  as  did  reorganization.  In  addition,  all  ALCs 
experienced  a  workload  reduction  during  1987;  although  the  other 
ALCs  rebounded,  Sacramento’s  workload  remained  lower  through 
June  1990,  the  end  point  of  our  analysis.  Despite  turbulence  and 
change,  our  analysis  reveals  substantial  similarity  across  ALCs  in 
factors  affecting  labor  cost.  This  applies  both  to  the  baseline  and  the 
demonstration  periods. 

Three  findings  form  the  core  of  our  results.  First,  although 
Sacramento  did  not  pay  gainshares  during  the  first  year  of  PACER 
SHARE,  gainshares  were  paid  in  all  but  one  quart^er  thereafter  from 
winter  1989  through  spring  1990. ^  For  gainshares  to  be  paid,  current 
unit  cost  must  be  less  than  “would  have”  unit  cost,  which  in  PACER 
SHARE  equals  baseline  average  unit  cost.  Thus,  the  pay-ment  of 
gainshares  offers  some  evidence  of  cost  savings  relative  to 
Sacramento’s  baseline  performance.  Moreover,  for  reasons  given  be¬ 
low,  we  believe  the  PACER  SHARE  gainsharing  formula  underesti¬ 
mated  the  size  of  gainshares  that  could  have  been  paid.  Second,  when 
analyzed,  those  cost  savings  were  nevertheless  not  statistically  signif¬ 
icant.  This  holds  despite  the  five-of-six-quarter  string  of  gainshare 
payments.  If  Sacramento’s  recent  performance  continues,  however, 
the  greater  accumulation  of  data  might  enable  us  to  discern  signifi¬ 
cant  cost  savings.  As  it  is,  the  gainshare-related  cost  savings  lie 


lA  productivity  cash  award  was  paid  in  the  first  quarter  of  the  first  year,  but  it  was 
not  based  on  the  productivity  gainsharing  formula.  See  App.  B,  Table  B.5,  for  a  listing 
of  the  gainshares  paid  by  Sacramento. 
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within  the  range  of  values  expected  from  baseline  performance. 
Third,  as  in  previous  sections  we  place  most  weight  on  Sacramento’s 
performance  compared  with  that  of  other  ALCs.  The  other  ALCs  re¬ 
flect  systemic  patterns  one  might  expect  of  Sacramento  without 
PACER  SHARE.  In  fact,  their  cost  moved  similarly  to  Sacramento’s, 
so  by  this  comparative  yardstick  Sacramento  again  showed  no  statis¬ 
tically  significant  cost  savings. 

This  section  reports  the  underlying  analysis,  focusing  first  on  cost 
savings,  then  on  gainshare  determination.  We  describe  how  the  cur¬ 
rent  analysis  relates  to  the  year-one  analysis,  review  the  labor  cost 
model,  discuss  hypotheses  to  be  tested,  and  give  regression  results 
and  hypothesis  tests.  We  conclude  with  a  discussion  of  the  PACER 
SHARE  gainsharing  formula.^ 

RELATIONSHIP  TO  YEAR-ONE  COST  ANALYSIS 

Our  capability  to  evaluate  cost  savings  under  PACER  SHARE  has 
expanded  significantly  since  last  year.  We  have  another  year’s  worth 
of  demonstration  period  data  and,  equally  important,  now  have  com¬ 
plete  period  data  for  all  five  ALCs.  Further,  output  data  come  from 
the  financial  data  system,  which,  because  of  its  auditability  and  strict 
one-to-one  correspondence  with  actual  issues  and  receipts,  surpasses 
the  workload  data  system.  For  the  first-year  analysis,  by  comparison, 
we  had  partial  data  for  two  ALCs — San  Antonio  and  Oklahoma 
City — and  had  to  omit  them  from  analysis.  Also,  because  we  were 
limited  to  comparisons  between  Sacramento  and  the  pair  Ogden  and 
Warner-Robins,  we  conducted  parallel  analyses  with  output  data  from 
the  financial  system  and  the  less-preferred  workload  system,  hoping 
for  robust  results.  The  different  data  did  not  always  produce  the 
same  insights,  and  there  was  no  good  way  to  resolve  the  differences. 
Having  more  of  the  better  data  avoids  this  problem. 

The  expanded  database  permits  us  to  specify  and  estimate  a  more 
general  model  than  before.  We  have  taken  advantage  of  this  oppor¬ 
tunity  in  several  ways.  First,  using  this  model  we  found  that  neither 
of  our  previous  models — one  with  unit  cost  as  dependent  variable,  the 


^That  explanation  discusses  the  current  gainsharing  formula,  which  differs  from 
the  one  initially  used.  The  original  model  determined  baseline  productivity  through 
calculations  that  used  earned  hours,  actual  hours,  and  total  costs.  After  evaluating 
various  systems,  however,  it  was  determined  that  a  measurement  system  based  on  unit 
or  transaction  costs  would  more  accurately  reflect  actual  savings.  The  quarterly  trans¬ 
action  costs  for  1986  through  1987,  converted  to  current  dollars,  were  averaged  to  de¬ 
termine  the  baseline  transaction  cost.  The  change  is  discussed  in  the  March  30,  1990, 
Federal  Register. 
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other  with  total  labor  cost  as  dependent  variable  but  with  a  more  re¬ 
strictive  specification  of  the  time  variable — was  entirely  satisfactory. 
Second,  we  now  test  hypotheses  on  whether  intercept,  time  trend,  and 
output  effect  taken  as  a  set  and  individually  differ  between 
Sacramento  and  the  other  ALCs  and,  for  Sacramento  and  the  other 
ALCs,  differ  between  baseline  and  demonstration  periods. 
Previously,  with  our  more  limited  models,  hjrpothesis  testing  was 
confined  to  asking  whether  the  time  trend  effects  differed.  Third,  we 
now  have  enough  data  to  explore  whether  the  other  ALCs  are  suffi¬ 
ciently  similar  to  share  the  same  cost  structure,  as  had  been  assumed, 
or  whether  our  results  depend  on  an  ALC’s  aberrant  behavior.  We  do 
this  by  estimating  our  model  separately  for  each  ALC — Oklahoma 
City,  Ogden,  San  Antonio,  and  Warner-Robins — visually  comparing 
their  estimates  and  running  various  pooled-data  models.  Finally,  we 
test  whether  unit  labor  cost  depends  on  output  level.  Although 
PACER  share’s  gainsharing  formula  assumes  independence,  we 
find  dependence.  We  discuss  the  consequences  of  this  disparity  for 
the  gainshare  computation. 

LABOR  COST  MODEL 

We  estimate  labor  cost  as  a  function  of  an  intercept,  time,  and  output, 
as  explained  in  Sec.  2.  The  intercept  controls  for  fixed  factors  that  can 
affect  cost,  such  as  plant,  equipment,  theaters  served,  weapons  sys¬ 
tems  supported,  work  force  skill  and  experience,  and  work  force 
turnover.  These  factors  are  “fixed”  in  the  short  term  but  may  change 
over  a  longer  period.  Change  could  result  from  factors  felt  throughout 
the  AFLC  system  or  ALC-specific  events  such  as  PACER  SHARE. 
Time  captures  the  net  influence  of  changes  cumulating  during  a  pe¬ 
riod  of  analysis.  This  includes  change  in  equipment,  training,  and 
procedures  needed  by  the  ALC  to  accomplish  its  materiel  receiving, 
storing,  and  shipping  functions.  It  also  includes  changes  in  the  size, 
skill  mix,  and  compensation  level  of  the  work  force.  Movement  to¬ 
ward  a  larger  work  force,  a  work  force  requiring  more  training,  or  a 
higher-paid  work  force  would  exert  upward  pressure  on  labor  cost 
over  time,  just  as  technical  change,  multiple  skill  training,  and 
streamlined  procedures  might  exert  downward  pressure.  Controlling 
for  the  fixed  factors  and  the  time  trend,  the  output  effect  describes  the 
percentage  change  in  labor  cost  per  percent  change  in  output.  A 
higher  workload  demands  more  labor,  hence  labor  cost  should  rise  as 
output  increases.  But  a  1  percent  increase  in  output  likely  requires  a 
less- than- 1-percent  increase  in  labor  cost  because  incumbent  workers 
often  can  work  more  intensively,  and  because  added  workers  typically 
receive  lower  wages. 
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The  intercept,  time  trend,  and  output  effect  are  allowed  to  differ  be¬ 
tween  baseline  and  demonstration  periods,  for  both  Sacramento  and 
comparison  ALCs.  We  estimate  not  only  the  parameters  for  each  pe¬ 
riod  but  also  the  intercepts.  Therefore,  although  the  labor  cost  model 
has  a  simple  structure,  it  can  detect  differences  between  Sacramento 
and  comparison  ALCs  within  a  period,  and  differences  for  each  be¬ 
tween  periods.  The  model’s  basic  structure  for  an  ALC  in  a  period  is: 


^n  Cj  -  ao  +  a^  t  +  a2  fn  Xj  +  e^. 


Here,  fn  c^  is  log  labor  cost  in  month  t,  ag  is  the  intercept,  t  is  the 
time  variable  measured  in  months,  a^  is  the  time  trend  (i.e.,  propor¬ 
tionate  change  in  labor  cost  per  month),  fn  Xj  is  log  output  measured 
as  total  issues  and  receipts  in  month  t,  and  aj  is  the  output  effect  (i.e., 
percentage  change  in  labor  cost  per  percentage  increase  in  output). 

HYPOTHESES  TO  TEST 

If  PACER  SHARE  interventions  are  effective  in  improving  organiza¬ 
tional  flexibility,  so  that  resources  may  be  reallocated  more  rapidly 
and  efficiently,  and  effective  in  creating  incentives  to  encourage  those 
reallocations,  then  Sacramento’s  cost  should  decline  relative  to  that  of 
other  ALCs.  Such  a  cost  decline  can  result  from  lower  cost  at 
Sacramento  relative  to  its  base  period  cost,  or  higher  cost  at  other 
ALCs  relative  to  their  base  period  cost,  or  of  course  lower  cost  both 
places  but  a  greater  cost  decline  at  Sacramento.  Interestingly,  the 
pattern  of  cost  changes  may  affect  the  success  of  PACER  SHARE. 
Under  the  gainsharing  formula,  gainshares  may  be  paid  if 
Sacramento’s  unit  labor  cost  declines  relative  to  its  baseline  unit  cost, 
and  gainshares  themselves  may  spur  Sacramento  to  attain  further 
cost  reductions.  But  if  unit  cost  does  not  decline  at  Sacramento,  gain- 
shares  will  not  be  paid  and  the  feedback  effect  will  not  operate.  There 
could  even  be  an  adverse  effect  if  workers  expect  gainshares  but  re¬ 
ceive  none.  Our  evaluation  methodology  nevertheless  could  show 
Sacramento’s  cost  declining  relative  to  that  of  the  other  ALCs. 

In  evaluating  cost  savings,  we  are  most  interested  in  whether 
Sacramento’s  cost  change  from  base  period  to  demonstration  period 
was  less  than  the  respective  change  at  other  ALCs.  If  so,  we  infer 
that  Sacramento  achieved  cost  -ings  relative  to  what  would  have 
been  expected  without  PACER  SHARE.  In  addition  to  the  relative 
change  hypothesis,  we  test  a  series  of  intermediate  hypotheses.  All 
the  hypotheses  implicitly  compare  the  cost  of  producing  a  given  level 
of  output  at  a  given  time.  This  qualification  is  necessary  because  cost 
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generally  depends  on  time  and  output  level,  that  is,  c  =  c(t,  x),  and  un¬ 
less  t  and  X  are  given  the  cost  comparisons  are  ambiguous.  For  this 
reason  it  is  not  suitable  simply  to  test  average  costs  in  the  base  period 
versus  those  in  the  demonstration  period;  these  costs  are  unadjusted 
for  time  trend  and  output  level.  We  use  regression  analysis  to  make 
the  adjustments.  Finally,  for  each  of  the  cost  hypotheses  we  also  test 
three  associated  hypotheses  concerning  the  intercept,  time  trend,  and 
output  effect;  they  help  isolate  the  sources  of  cost  differences,  if  any. 

Hypothesis  1:  Cost  is  the  same  at  Sacramento  and  other  ALCs  during 
the  base  period.  We  expect  this  hypothesis  to  be  rejected  because  of 
intrinsic  differences  in  the  ALCs  reflecting  different  physical  layouts 
and  workload  mixes  (beyond  what  we  can  measure  with  our  output 
data).  The  workload  mix  depends  on  the  mission  assigned  to  the  ALC 
such  as  which  theaters  and  weapons  systems  the  ALC  primarily  sup¬ 
ports. 

Hypothesis  2:  Cost  is  the  same  at  other  ALCs  in  the  demonstration 
period  as  in  the  base  period.  By  looking  at  other  ALCs  we  learn  what 
happened  within  the  system  and  hence  what  would  be  expected  to 
happen  at  Sacramento  were  PACER  SHARE  not  introduced.  Cost  at 
the  other  ALCs  may  have  declined,  remained  constant,  or  risen  rela¬ 
tive  to  the  base  period.  Whatever  the  pattern,  it  is  the  backdrop  for 
judging  Sacramento. 

Hypothesis  3:  Cost  is  the  same  at  Sacramento  in  the  demonstration 
period  as  in  the  base  period.  If  PACER  SHARE  increases  productiv¬ 
ity,  then  cost  in  the  demonstration  period  should  be  lower  than  at 
baseline  and  the  hypothesis  rejected.  As  mentioned,  a  cost  reduction 
must  occur  for  the  payment  of  gainshares  at  Sacramento.  Still,  this 
hypothesis  is  an  incomplete  test  of  PACER  SHARE’S  effect  on  cost  be¬ 
cause  it  neglects  what  is  happening  at  other  ALCs. 

Hypothesis  4;  Cost  is  the  same  at  Sacramento  and  other  ALCs  during 
the  demonstration  period.  A  counterpart  to  hypothesis  1,  this  hypoth¬ 
esis  could  be  rejected  because  of  differences  in  mission  and  physical 
layout.  Still,  it  is  possible  Sacramento’s  and  other  ALCs’  costs 
changed  in  such  a  way  as  to  make  them  equivalent  during  the  demon¬ 
stration. 

Hypothesis  5;  Sacramento’s  cost  change  from  baseline  to  demonstra¬ 
tion  period  is  the  same  as  other  ALCs’  cost  change  from  baseline  to 
demonstration  period.  This  hypothesis  incorporates  information  from 
all  previous  hypotheses  and  provides  a  comparative  assessment  of 
Sacramento’s  performance  under  PACER  SHARE.  We  might  find 
Sacramento’s  cost  declined  and  did  so  more  rapidly  than  other  ALCs’; 
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or  we  might  find  Sacramento’s  cost  did  not  decline  but  other  ALCs’ 
cost  rose — either  pattern  would  indicate  cost  savings  under  PACER 
SHARE  and  the  hypothesis  would  be  rejected.  If  the  hypothesis  is  not 
rejected,  there  is  no  statistically  significant  evidence  of  PACER 
SHARE  cost  savings. 

REGRESSION  RESULTS 

To  implement  our  approach  we  needed  to  determine  a  suitable  way  of 
pooling  data  on  other  ALCs  for  the  purpose  of  comparing 
Sacramento’s  cost  with  theirs.  We  sought  a  pooled-data  model  that 
reflects  “typical”  ALC  behavior,  and  we  chose  tu  judge  typicality  by 
referring  to  labor  cost  equations  estimated  for  other  ALCs  individu¬ 
ally.  We  began  by  estimating  the  regressions  by  ALC,  then  consid¬ 
ered  various  alternatives  for  a  pooled-data  regression.  During  these 
steps  we  found  Oklahoma  City’s  results  to  be  anomalous  and  decided 
to  drop  the  ALC  from  the  comparison  group.  We  pursued  two  pooled- 
data  models,  one  having  a  single  intercept,  time  trend,  and  output 
effect  in  each  period,  and  the  other  having  an  intercept  for  each  in¬ 
cluded  ALC  (Ogden,  San  Antonio,  Warner-Robins)  but  a  single  time 
trend  and  output  effect  in  each  period.  We  prefer  the  latter  model  be¬ 
cause  of  its  better  fit  to  the  data  and  closeness  to  the  individual  ALC 
results.  However,  for  completeness  we  discuss  the  former  model  in 
App.  B;  we  also  include  the  labor  cost  and  output  data  by  ALC  in  App. 
C.  Below  we  give  relevant  data  plots,  labor  cost  regressions  by  ALC, 
the  multiple-intercept  pooled  model  results  versus  Sacramento’s,  and 
the  hypothesis  tests. 

Figure  1  displays  labor  cost  and  output  over  time  by  ALC.  The  figure 
shows  a  close  correspondence  between  those  variables,  suggesting 
that  the  labor  cost  model  will  perform  adequately  at  the  ALC  level. 
Labor  cost  tracks  output  equally  well  in  the  demonstration  and  base 
periods,  so  the  model  should  be  effective  in  both  periods.  An  excep¬ 
tion  is  Oklahoma  City,  which  has  a  weaker  association  between  out¬ 
put  and  labor  cost  during  baseline.  Also,  as  seen,  Sacramento’s  out¬ 
put  fell  and  remained  low,  while  other  ALCs’  output  fell  and 
rebounded.  These  patterns  pertain  especially  to  the  gainsharing  dis¬ 
cussion  below. 

Labor  cost  regressions  for  each  ALC  appear  in  Table  31.  We  found  ev¬ 
idence  of  similarity  across  the  ALCs  in  both  periods,  supporting  the 
case  that  a  regression  based  on  other  ALCs’  pooled  data  will  accu¬ 
rately  represent  the  system,  which  we  seek  for  comparison  with 
Sacramento.  In  the  baseline  period  we  see  (1)  sim.ilar  intercepts  with 
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Table  31 


Labor  Cost  Regressions  by  ALC 
(dependent  variable:  labor  cost) 


Variable 

Sacramento 

Oklahoma 

City 

Ogden 

San 

Antonio 

Wamer-Robins 

BASELINE  PERIOD 

Intercept 

11.28 

12.04 

11.67 

13.43 

13.55 

(2.86) 

(2.03) 

f2.13) 

(2.50) 

(2.71) 

Time 

-.0019 

.0008 

-.0009 

.0009 

-.0011 

(.0014) 

(.0010) 

(.0012) 

(.0012) 

(.0012) 

/n  output 

.36 

.28 

.33 

.18 

.18 

(.24) 

(.17) 

(.18) 

(.21) 

(.22) 

DEMONSTRATION  PERIOD 

Intercept 

13.81 

4.08 

11.57 

10.20 

12.84 

(2.69) 

(2.30) 

(2.18) 

(2.17) 

(2.32) 

Time 

-.0026 

.0091 

.0000 

.0015 

-.0001 

(.0019) 

(.0016) 

(.0018) 

(.0019) 

(.0017) 

/n  output 

.14 

.90 

.33 

.43 

.22 

(.23) 

(.19) 

(.18) 

(.18) 

(.19) 

Act).  R-bquare 

.77 

.16 

.53 

.43 

.56 

Standard  error 

of  estimate 

.06 

.11 

.07 

.05 

.06 

NOTE:  Standard  errors  are  in  parentheses. 


values  ranging  from  11.57  to  13.55,  all  within  roughly  one  standard 
error  of  each  other,  (2)  seemingly  different  time  trends  but  all  within 
two  standard  errors  of  one  another,  and  (3)  output  coefficients  rang¬ 
ing  from  .18  to  .33,  yet  again  within  one  standard  error  of  each  other. 
The  same  points  hold  across  the  other  ALCs  in  the  demonstration  pe¬ 
riod  (except  for  Oklahoma  City)  and  also  between  the  base  period  and 
demonstration  period.  Indeed,  Oklahoma  City’s  demonstration  period 
coefficients  and  its  low  R-square  (poorer  fit)  are  an  exception  to  the 
pattern. 

We  next  estimated  pooled-data  models  for  all  other  ALCs.  Generally, 
Oklahoma  City  exerted  a  strong  influence,  making  the  results  atypi¬ 
cal  of  the  ALCs.  A  single-intercept  model  with  Oklahoma  City  is  re¬ 
ported  in  Table  B.2,  and  we  also  flt  multiple-intercept  models  includ¬ 
ing  Oklahoma  City.  Based  on  these  pooled  regressions  as  well  as 
Oklahoma  City’s  own  estimates,  we  deleted  Oklahoma  City  from  the 
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pooled  data.  Even  though  Oklahoma  City  is  absent,  in  what  follows 
we  refer  to  the  remaining  threesome  as  “other  ALCs.” 

Motivation  for  the  multiple-intercept  model  over  the  single-intercept 
model  comes  from  Fig.  2,  which  plots  ^n  labor  cost  against  ^n  output. 
The  separate  clusters  of  points  by  ALC  have  similar  orientations, 
providing  justification  for  assuming  a  common  output  effect.  The 
clusters  do  not  overlay  one  another,  however,  but  are  shifted  left  or 
right,  indicating  different  output  ranges.  Given  the  similar  cluster 
shapes  but  different  locations,  a  model  with  a  common  output  effect 
and  multiple  intercepts  seems  well  suited.  Further,  judging  from  Fig. 
1  (showing  labor  cost  by  time)  and  in  view  of  the  regressions  by  ALC, 
it  is  reasonable  to  specify  a  common  time  trend. 

In  contrast  to  the  multiple-intercept  model,  the  single-intercept  model 
treats  the  pooled  cluster  of  points  as  a  unit.  As  Fig.  2  shows,  the  ori¬ 
entation  of  the  pooled  cluster  differs  from  the  ALC  clusters.  The  dif¬ 
ference  is  subtle  at  baseline  but  pronounced  in  the  demonstration 
period,  where  the  pooled  cluster  has  an  elongated  shape  and  more 
horizontal  orientation.  It  is  therefore  not  surprising  that  the  single¬ 
intercept  model  has  a  high  intercept  and  low  output  effect  in  the 
demonstration  period  (see  Table  B.l),  with  values  lying  outside  the 
individual  ALC  range. 

The  multiple-intercept  regression,  shown  in  Table  32,  appears  repre¬ 
sentative  of  the  other  ALCs  and  consequently  affords  a  good  basis  for 
hypothesis  tests.  The  regression  fits  the  data  well,  displaying  fidelity 
to  the  individual  ALC  results  given  before.  Notably,  each  coefficient 
lies  in  mid  range  of  the  latter,  which  is  not  true  of  the  single-intercept 
model. 

As  compared  with  the  baseline  period,  during  the  demonstration 
period  the  other  ALCs’  intercepts  decline  by  roughly  a  unit,  the  time 
trend  changes  from  a  small  negative  trend  to  a  small  positive  one, 
and  the  output  effect  rises.  In  contrast,  Sacramento’s  intercept 
increases  from  baseline  to  demonstration  period,  the  time  trend 
becomes  a  stronger  negative  effect,  and  the  output  effect  falls. 
Although  Sacramento’s  and  other  ALCs’  coefficients  change  between 
the  baseline  and  demonstration  periods,  the  changes  fall  within  one 
or,  for  the  time  trend,  two  standard  errors.  These  are  not  large 
differences  from  a  statistical  perspective. 


□  □ 


i’n  output 


Fig.  2 — {n  Labor  Cost  vs.  fn  Output  by  ALC 
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Table  32 


Labor  Coat  Regressions:  Sacramento 
vs.  Comparison  Group 
(dependent  variable:  tn  labor  cost) 


Variable 

Sacramento 

Other  ALCb 

BASELINE  PERIOD 

Intercept 

Sacramento 

11.28 

Ogden 

(2.86) 

12.26 

San  Antonio 

(1.25) 

12.21 

Wamer-Robina 

(1.26) 

12.30 

Time 

-.0019 

(1.26) 

-.0004 

(.0014) 

(.0006) 

in  output 

.36 

.28 

(.24) 

(.10) 

DEMONSTRATION  PERIOD 

Intercept 

Sacramento 

13.81 

Ogden 

(2.69) 

11.27 

San  Antonio 

(1.16) 

11.19 

Wamer-Robina 

(1.18) 

11.25 

Time 

-.0026 

(1.17) 

.0006 

(.0019) 

(.0009) 

in  output 

.14 

.35 

(.23) 

(.10) 

Adj.  R-square 

.77 

.41 

Standard  error  of 

eatimate 

.06 

.08 

NOTE:  Standard  errors  are  in  parentheses. 


HYPOTHESIS  TEST  RESULTS 

We  present  hypothesis  test  results  in  Table  33.  The  tests  are  based 
on  a  pooled-data  regression  rather  than  on  the  separate  regressions 
reported  in  Table  32.  Pooling  the  data  ensures  a  common  estimated 
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Table  33 

Hypothesis  Test  Results  Using  Warner-Robins  Intercept 


HI  hypotheses:  Sacramento  versus  other  ALCs  during  baseline  period. 

fn  Cost 

Intercept 

Time  Trend 

fn  Output 

Equal?  Reject 

Accept 

Accept 

Accept 

Probability  .0001 

.71 

.26 

.73 

H2  H3rpotheses:  Other  ALCs,  demonstration  versus  baseline  periods. 

fn  Cost 

Intercept 

Time  Trend 

fn  Output 

Equal?  Reject 

Accept 

Accept 

Accept 

Probability  .0013 

.52 

.38 

.58 

H3  Hypotheses:  Sacramento,  demonstration  versus  baseline  periods. 

fn  Cost 

Intercept 

Time  Trend 

(n  Output 

Equal?  Accept 

Accept 

Accept 

Accept 

Probability  .21 

.45 

.69 

.44 

H4  Hypotheses;  Sacramento  versus  other  ALCs  during  demonstration 

period. 

(n  Cost 

Intercept 

Time  Trend 

<n  Output 

Equal?  Reject 

Accept 

Reject 

Accept 

Probability  .0001 

.31 

.08 

.32 

H5  Hypotheses:  Change  at  Sacramento,  demonstration  versus  baseline 
periods,  versus  change  at  other  ALCs,  demonstration 
versus  baseline  periods. 

tn  Cost  Intercept  Time  Trend  tn  Output 
Equal?  Accept  Accept  Accept  Accept 

Probability  .68  .34  .44  .35 


error  variance.  The  pooled  regression  gives  the  same  coefTicients  as 
appear  in  Table  32,  but  the  standard  errors  are  slightly  different  be¬ 
cause  of  the  pooling.  Given  the  high  degree  of  similarity  in  the  stan¬ 
dard  errors  in  the  separate  versus  pooled  models,  either  approach 
would  produce  virtually  the  same  hypothesis  test  results.  We  use  the 
pooled  regression  because  a  common  error  variance  simplifies  com¬ 
puting  the  tests.3  As  mentioned,  we  test  for  equality  of  labor  cost  be¬ 
tween  Sacramento  and  other  ALCs  during  baseline  and  during  the 
demonstration  period,  equality  at  Sacramento  between  periods. 


3ln  estimating  the  pooled  model  we  also  tested  for  autocorrelation.  Specifically,  we 
estimated  models  allowing  for  (1)  first-order  autocorrelation  and  (2)  first-  through 
sixth-order  autocorrelation.  In  both  cases  the  estimates  and  standard  errors  changed 
very  little  from  the  basic  model,  assuming  no  autocorrelation.  The  Durbin-Watson 
statistic  for  (1)  was  1.65,  indicating  only  borderline  autocorrelation  at  most.  As  a  result 
of  this  exercise,  we  are  confident  that  the  hypothesis  test  results  are  robust  to  autocor¬ 
relation;  had  we  used  the  results  from  (1)  or  (2)  the  hypothesis  test  results  would  have 
been  the  same  as  obtained  for  the  basic  model,  whose  results  are  reported  in  Table  33. 
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equality  at  other  ALCs  between  periods,  and  equality  of  cost  change 
at  Sacramento  relative  to  that  at  other  ALCs.  In  addition,  we  conduct 
similar  tests  for  intercept,  time  trend,  and  output  effect  estimates. 
Because  we  use  a  multiple-intercept  model,  we  also  must  choose 
which  intercept  to  use  in  the  tests.  In  fact,  we  performed  separate 
sets  of  tests  using  each  of  the  three  intercepts  and  obtained  much  the 
same  results.  For  present  purposes  it  is  therefore  enough  to  present 
one  set  of  results,  which  we  do  u^'  ig  Warner-Robins’  intercept. 
(Other  test  results  are  available  on  reqf.est.) 

Tests  of  cost  h3q5otheses  ask  whether  after  controlling  for  time  and 
output,  two  costs  are  statistically  equivalent.  For  example,  hypothe¬ 
sis  1  asks  whether  Sacramento  and  other  ALCs  have  the  same  cost  at 
baseline:  if  c(SM,  B;  t,  x)  and  c(Other,  B;  t,  x)  represent  baseline 
costs,  are  the  two  costs  equal  for  a  given  t  and  x?  To  control  for  t  and 
X  one  can  use  the  regression  coefficients  to  predict  costs  at  any  given  t 
and  X,  obtaining  estimates  of  what  costs  would  have  been  had 
Sacramento  and  other  ALCs  produced  the  same  output  at  the  same 
time.  But  making  such  predictions  is  unnecessary.  Because  t  and  x 
are  the  same  in  these  predictions,  any  difference  in  predicted  costs 
necessarily  comes  from  the  regression  coefficients.  Thus  the  test  for 
cost  equality  reduces  to  a  test  of  equality  between  Sacramento’s  and 
other  ALCs’  intercepts,  time  trends,  and  output  effects,  taken  as  a  set. 
The  F-statistic  is  appropriate  for  this  test,  just  as  for  tests  of  equality 
between  individual  coefficients. 

Entries  in  Table  33  state  whether  the  hypothesis  is  rejected  or  not 
(“accepted”)  and  give  the  probability  of  the  F-statistics  in  the  particu¬ 
lar  test,  A  value  of  .26,  for  instance,  means  that  the  chance  of  observ¬ 
ing  a  value  of  the  test  statistic  greater  than  that  computed  in  the  test 
is  26  percent,  assuming  the  hypothesis  is  true.  In  other  words,  the 
outcome  would  be  moderately  common  and  the  results  are  consistent 
with  the  hypothesis.  The  value  of  .26  is  well  above  usual  levels  of 
statistical  significance  for  rejecting  an  hypothesis,  which  are  perhaps 
.10  or,  more  typically,  lower  (.05  or  .01).  On  the  other  hand,  if  the 
probability  of  the  test  statistic  is  below  the  significance  level,  the  hy¬ 
pothesis  is  rejected. 

The  results  of  the  hypothesis  tests  are  consistent  with  the  following 
statements:  (1)  Sacramento’s  intercept,  time  trend,  and  output  effect, 
when  taken  individually,  are  the  same  as  other  ALCs’  at  baseline. 
However,  when  the  coefficients  are  viewed  as  a  group,  Sacramento 
differs  from  other  ALCs.  This  occurs  because  the  sets  of  coefficients 
are  more  precisely  measured  than  are  single  coefficients.  Still,  the 
results  suggest  that  Sacramento  and  the  other  ALCs  are  reasonably 
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comparable  at  baseline,  thereby  providing  a  foundation  for  the  further 
hypothesis  tests.  (2)  Other  ALCs’  cost  differs  between  baseline  and 
demonstration  periods.  The  factors  driving  this  difference  are  (non¬ 
significant)  changes  in  the  intercept,  time  trend,  and  output  effects. 
The  intercept  becomes  smaller  and  the  output  effect  larger,  for 
instance.  (3)  Sacramento’s  coefficients  also  change,  but  on  net  not  by 
enough  to  result  in  a  significant  cost  difference  between  baseline  and 
demonstration  periods.  This  result  partly  reflects  the  smaller  preci¬ 
sion  of  Sacramento’s  estimates  due  to  its  smaller  sample  size  than  the 
other  ALCs’  pooled-data  sample.  (4)  Sacramento’s  cost  differs  from 
other  ALCs’  cost  in  the  demonstration  period.  The  difference  in  time 
trend  undoubtedly  contributes  to  this  result — Sacramento’s  trend  is 
-.0026,  other  ALCs’,  .0006.  (5)  The  change  in  Sacramento’s  cost  from 
baseline  to  demonstration  periods  is  the  same  as  the  change  in  other 
ALCs’  cost.  This  is,  of  course,  the  most  important  hypothesis  test.  By 
implication  of  statements  (3)  and  (5),  the  results  indicate  no  statisti¬ 
cally  significant  PACER  SHARE  cost  savings,  judging  either  from 
Sacramento’s  performance  under  PACER  SHARE  as  compared  with 
its  previous  performance,  or,  as  our  approach  emphasizes,  the  change 
in  its  performance  relative  to  that  of  other  ALCs. 

Nevertheless,  the  gainsharing  discussion  below  presents  figures 
based  on  the  regression  analysis  that  reveal  a  tendency  toward  cost 
savings  at  Sacramento  relative  to  its  baseline.  Sacramento’s  cost  sav¬ 
ings,  although  not  (yet)  statistically  significant,  may  therefore  be  in¬ 
cipient.  On  the  other  hand,  ve  find  a  s.milar  pattern  of  potential  cost 
savings  at  other  ALCs.  Therefore,  under  PACER  SHARE  Sacramento 
may  be  outperforming  its  past,  and  other  ALCs  without  PACER 
SHARE  may  be  outperforming  their  past  too. 

GAINSHARING 

PACER  SHARE’S  gainsharing  formula  determines  the  money  avail¬ 
able  for  the  gainshare  pool  (.gf) — half  of  which  is  returned  to  the  Air 
Force — as  a  function  of  the  difference  between  baseline  and  current 
unit  cost  times  current  output; 

gt  -[(co/xo)-(ct/xt)](xt)  if  right  hand  side  >  0 
-  0  otherwise 

The  gainshare  pool  is  positive  if  current  unit  cost  is  less  than  baseline 
unit  cost.  If  current  unit  cost  exceeds  or  equals  baseline  unit  cost,  the 
gainshare  is  zero.  The  formula  thus  rewards  improved  productivity 
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(lower  unit  cost)  with  respect  to  Sacramento’s  baseline  without  penal¬ 
izing  reduced  productivity.  The  formula  is  easily  implemented  and 
draws  data  from  the  financial  system,  the  same  data  system  we  use. 

The  formula’s  simplicity  engenders  a  deficiency,  however.  The  for¬ 
mula  should  conceptually  depend  on  the  difference  between  what 
would  have  been  the  unit  cost  of  current  output  and  what  it  was,  but 
in  practice  “would  have”  unit  cost  equals  a  single  number,  average 
baseline  unit  cost.  This  approach  rules  out  the  possible  dependence  of 
unit  cost  on  level  of  output. 

Whether  unit  cost  depends  on  output  cannot  be  readily  inferred  from 
tables  or  plots  routinely  available  to  ALC  management,  namely,  data 
on  unit  cost  by  month.  Such  data,  plotted  in  Fig.  3,  show  little  time 
trend  in  unit  cost  except  for  Oklahoma  City  in  the  de’^  .tration  pe¬ 
riod.  Even  bringing  to  mind  the  output  plot  by  m  nth  given  in  Fig.  1, 
the  eye  cannot  determine  whether  and  to  .vhat  extent  unit  cost  de¬ 
pends  on  output  level.  The  unit  cost  plots  offer  no  reason  not  to  de¬ 
fine  a  gainsharing  formula  where  “would  have”  cost  is  independent  of 
output. 

Our  model,  in  contrast,  can  be  used  to  determine  the  presence  and  ex¬ 
tent  of  dependency.  The  labor  cost  model  is: 

fn  Ct  -  ag  +  ai  t  +  33  in  Xj  +  e^ 


Subtracting  in  x^  from  both  sides  gives  an  average  cost  equation: 
in  (ct  /  Xt )  =  ag  +  ajt  +  (33  -  l)  in  Xj  + 


Here,  a  1  percent  increase  in  x  relates  to  an  (33  -  1)  percent  change  in 
unit  cost,  c/x.  We  can  use  the  estimated  coefficient  83  to  test  the  inde¬ 
pendence  assumption:  if  (33  -  1)  differs  significantly  from  zero,  the 
independence  assumption  is  rejected. 

The  intuition  behind  the  importance  of  adjusting  for  the  dependence 
of  unit  cost  on  output  level  can  be  described  by  differentiating  the 
gainshare  equation  with  respect  to  output.  (Readers  not  interested  in 
this  detail  may  want  to  skip  ahead.)  We  allow  “would  have”  unit  cost 
first  to  depend  on  output,  then  assume  it  does  not.  In  the  first  case: 


gt  - 


(cg/Xg)  -(Ct/Xt)  ](Xt)  +  [(co/Xg)-(Ct/xj] 
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The  first  term  in  brackets  on  the  right-hand  side  is  the  indirect  effect 
of  a  change  in  output  as  it  is  felt  through  “would  have”  unit  cost  and 
current  unit  cost,  respectively.  If  the  production  structure  and  effi¬ 
ciency  remain  largely  the  same,  these  incremental  effects  on  the  unit 
costs  would  be  nearly  zero,  and  as  a  result  the  first  term  would  van¬ 
ish.  The  second  term  is  the  direct  effect  of  a  change  in  output  and 
equals  the  difference  between  “would  have”  and  current  unit  cost. 

In  the  second  case,  representative  of  the  PACER  SHARE  gainshare 
formula,  “would  have”  unit  cost  does  not  vary  with  output,  hence  the 
effect  of  output  on  it  is  constrained  to  be  zero.  The  above  equation  be¬ 
comes: 


(xt)  +  [(co/Xo)-(ct/Xt)] 


The  difference  between  the  constrained  and  unconstrained  deriva¬ 
tives  is  the  term  -  (cq  /  Xg)  '(Xt).  Our  regression  results  show  that 
unit  cost  falls  as  output  increases,  so  the  unit  cost  derivative  in  this 
term  will  be  negative  in  reality  and  the  overall  term  will  be  positive. 
Therefore  the  gainshare  as  output  increases  will  be  larger  under  the 
PACER  SHARE  formula  than  under  the  more  general  case  where 
“would  have”  unit  cost  is  adjusted  for  change  in  the  level  of  output. 
For  Sacramento,  which  experienced  a  rapid  and  substantial  decline  in 
its  workload  just  before  PACER  SHARE  began,  it  follows  that  PACER 
SHARE  gainshares  were  below  what  they  would  have  been  had  out¬ 
put  level  remained  the  same.^ 

When  we  test  for  independence  of  unit  cost  from  output,  the  results 
clearly  reject  it  (Table  34).  Sacramento’s  regression  results  imply 
that  a  1  percent  increase  in  output  is  associated  with  about  a  .6  per¬ 
cent  drop  in  unit  cost  at  baseline  and  a  .8  percent  drop  during  demon- 


“•A  rough  idea  of  this  decrease  may  be  obtained  as  follows.  From  the  unit  cost 
equation 

d  fn  (c/x)/d  fn  X  -{a2  -  l)”  --7  approx, 
so  d(c/x)/dx--.7(c/x)/x 

We  therefore  obtain  the  following  valuation; 

-(c/x)  x-.7(c/x) 

Actual  unit  cost  has  not  changed  much  at  Sacramento  from  baseline  to  demonstration 
periods  (Fig.  3),  hence  the  contribution  of  Kc/xlg  -  (c/x)^]  to  the  derivative  is  very  small, 
whereas  the  term  .7  (c/x)  makes  a  large  contribution  to  the  gainshare  derivative.  This 
analysis  is  borne  out  in  Table  35  and  Fig.  4,  showing  larger  gainshares  when  “would 
have”  unit  cost  is  acljusted  for  the  decrease  in  output  occurring  during  the  onset  of 
PACER  SHARE. 
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Table  34 

Tests  for  Independence  of  Unit  Cost  from  Output 


Site 

aj-l 

t-value 

Reject  Independence 

(ag  -  1)  =  0? 

BASELINE 

Sacramento 

-.64 

(.23) 

-2.8 

yes 

Other  ALCa  -.72 

(.10) 

DEMONSTRATION 

-7.2 

yes 

Sacramento 

-.86 

(.24) 

-3.6 

yes 

Other  ALCs 

-.65 

(.09) 

-7.2 

yes 

NOTE:  All  t-values  exceed  the  .01  significance  level  of  2.4. 
Estimates  of  82  come  from  Table  32.  Standard  errors  are  in 
parentheses. 


stration  period.  For  other  ALCs  the  response  is  about  .7  percent 
either  period,  not  much  different  than  Sacramento.  What  are  the  im¬ 
plications  for  the  gainsharing  computation? 

The  dependence  of  unit  cost  on  output  has  little  importance  for  gain- 
sharing  if  workload  had  remained  the  same  from  baseline  to  the 
demonstration  period.  But  Sacramento’s  workload  fell  by  20  percent 
(Fig.  1),  enough  to  increase  unit  cost  by  about  14  percent  if  we  use  an 
intermediate  .7  estimate  for  Sacramento’s  output  effect  on  unit  cost. 
Unit  cost,  in  other  words,  would  have  been  about  14  percent  higher 
because  of  the  output  change  alone — without  any  change  in  operating 
efficiency. 

Table  35  shows  how  the  dependence  of  unit  cost  on  output  affects  the 
gainshare  computation  for  Sacramento.  The  table  shows  production, 
cost,  and  gainshare  figures  based  on  setting  “would  have”  unit  cost 
equal  to  average  baseline  unit  cost  and,  alternatively,  with  it  equal  to 
ac^justed  baseline  unit  cost.  (The  computations  are  illustrative,  not 
official  gainshare  computations.)  The  first  two  columns  give  output 
and  labor  cost,  and  the  third  column,  the  ratio  of  labor  cost  to  output, 
or  unit  cost.  Next  come  two  versions  of  “would  have"  unit  cost;  base¬ 
line  average  unit  cost,  a  constant,  and  adjusted  baseline  unit  cost,  a 
quantity  predicted  from  Sacramento’s  baseline  regression  and  there- 


Table  35 
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fore  depending  on  time  and  output.®  These  alternative  versions  enter 
the  gainshare  formula  to  produce  unadjusted  and  adjusted  gainshare 
amounts:  gainshare  =  (“would  have”  unit  cost  -  current  unit  cost)  x 
current  output.  In  addition,  we  give  adjusted  “would  have”  unit  costs 
for  a  standard  error  above  and  below  the  regression  prediction,  and 
also  compute  gainshares  for  those  values.  Negative  values  are  shown 
in  parentheses. 

From  PACER  SHARE’S  viewpoint,  the  most  important  figures  in  the 
table  are  the  gainshares.  As  expected,  adjusting  for  output  level 
makes  a  msgor  difference  in  gainshare  size.  All  adjusted  gainshares 
are  positive  and  often  substantially  larger  than  the  unadjusted  gain- 
shares,  of  which  four  of  ten  are  negative.  The  adjusted  gainshares 
plus  a  standard  error  loom  still  larger,  of  course,  but  the  adjusted 
gainshares  minus  a  standard  error  are  less  than  the  unadjusted  gain- 
shares  about  half  the  time.  We  depict  the  gainshares  in  Fig.  4,  which 
shows  the  unadjusted,  adjusted,  and  adjusted-plus  and  -minus  one 
standard  error.  We  conclude  that  our  adjustment  produces  larger 
gainshares  not  only  for  our  best  estimate  (the  regression  prediction) 
but  also  for  a  reasonable  range  around  that  estimate.  Still,  at  two 
standard  errors  below  the  prediction  the  adjusted  gainshares  are  less 
than  zero,  a  fact  echoing  the  finding  of  no  statistically  significant  cost 
savings  at  Sacramento  relative  to  its  baseline. 

Although  the  adjusted  gainshares  exceed  the  unadjusted  gainshares, 
the  question  of  whether  paying  higher  gainshares  would  have  induced 
greater  productivity  remains  open.  We  frankly  expect  higher  gain- 
shares  to  create  stronger  feedback,  bringing  forth  more  effort,  creat¬ 
ing  higher  gainshares,  and  so  forth.  But  how  much  stronger  remains 
an  empirical  question.  The  effect  might  be  weak  if  gainshare  size 
bears  little  relationship  to  worker  effort,  a  possibility  given  the  per 
capita  distribution  of  gainshares.  Generally  speaking,  without  a  clear 


8As  in  the  regressions,  cost  figures  have  been  inflated  to  constant  1989  dollars;  see 
Sec.  2  for  inflation  factors.  Aiijusted  baseline  unit  cost  is  estimated  in  several  steps: 
monthly  log  unit  cost  is  predicted  from  the  baseline  unit  cost  regression  evaluated  at 
current  time  and  output  level,  that  is,  fn  (c^/x^)  =  11.28  -  .0019  t  -  .64fn  x*;  because 
we  assume  cost,  hence  unit  cost,  to  be  lognormally  distributed,  predicted  unit  cost 
equals  exp  (predicted  log  unit  cost  +  l/2s^),  where  s  is  the  standard  error  of  estimate, 
.057  for  Sacramento.  Predicted  monthly  unit  cost  is  then  averaged  on  a  quarterly  ba¬ 
sis.  Although  table  entries  for  unadjusted  gainshares  are  not  official,  they  are  close  to 
Sacramento’s  official  values.  They  differ  in  that  Sacramento  made  certain,  usually  mi¬ 
nor,  adjustments  to  current  average  cost  before  entering  it  in  the  gainshare  formula. 
However,  in  the  final  quarter  where  we  depict  a  negative  gainshare,  Sacramento  in  fact 
declared  a  positive  gainshare. 


Gainshare  pool 
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Fig.  4 — Sacramento  Gainshare  Pool  (Including  Air  Force  Share): 
Adjusted  vs.  Unadjusted 
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connection  between  effort  and  reward  workers  may  have  little  reason 
to  work  harder;  if  they  believed  in  such  a  connection  when  PACER 
SHARE  began,  their  belief  could  weaken  through  time  if  not  rein¬ 
forced. 

The  fact  that  gainshares  might  have  been  higher  at  Sacramento 
should  not  be  misconstrued.  We  have  found  no  statistically  signifi¬ 
cant  cost  savings,  and  our  model  inherently  accounts  for  the  depen¬ 
dence  of  unit  cost  on  output.  There  is  also  the  counterfactual  aspect: 
if  adjusted  gainshares  were  higher  at  Sacramento,  what  would  have 
happened  at  other  ALCs  had  they  been  under  gainsharing?  We  ad¬ 
dress  that  in  Tables  36  through  38  and  Figs.  5  through  7,  which  we 
present  chiefly  for  comparison  purposes  and  discuss  only  briefly.  The 
tables  and  figures  parallel  those  for  Sacramento.  For  each  other  ALC 
we  compute  gainshares,  unadjusted  and  adjusted,  and  depict  the  re¬ 
sults.  The  separate  computations  employ  the  intercepts  for  Ogden, 
San  Antonio,  and  Warner-Robins,  respectively,  and  the  time  trend 
and  output  effect  from  the  multiple  intercept  model.  The  results  for 
Ogden  and  Warner-Robins  are  similar  to  Sacramento’s,  as  one  would 
expect  given  our  hypothesis  tests  and  the  fact  that  they  also  had  a  de¬ 
cline  in  output  (Fig.  1).  For  San  Antonio,  in  contrast,  demonstration 
period  output  remained  at  baseline  level  on  average,  and  the  gain- 
share  adjustment  makes  little  difference.  In  all  cases,  the  adjusted 
calculation  results  in  positive  gainshares  in  nearly  all  quarters. 

SUMMARY 

We  have  studied  whether  Sacramento  achieved  labor  cost  savings 
under  PACER  SHARE  by  June  1990,  nearly  30  months  into  the 
demonstration.  By  comparison  with  the  year-one  analysis,  we  were 
fortunate  to  have  more  extensive  baseline  data  and  of  course  more 
demonstration  period  data.  The  added  data  allowed  us  to  estimate  a 
more  general  model  which  fit  the  data  better  than  our  previous  mod¬ 
els.  Data  plots  also  guided  our  modeling  choices.  In  developing  a  cost 
model  for  other  ALCs  for  the  purpose  of  making  comparisons  with 
Sacramento,  we  selected  a  specification  that  reflected  the  typical  be¬ 
havior  of  other  ALCs — the  multiple  intercept  model — but  for  com¬ 
pleteness  we  also  estimated  a  single  intercept  model  which,  we  found, 
led  to  the  same  conclusions. 

Using  the  regression  results,  we  performed  a  series  of  hypothesis 
tests  involving  Sacramento  and  other  ALCs  at  baseline  and  during 
the  demonstration  period.  We  inferred  from  the  results  that 
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Fig.  5— Ogden  Gainshare  Pool  (Including  Air  Force  Share) 
Adjusted  vs.  Unadjusted 


Gainshare  pool 


119 


$4,000,000 


$3,000,000 


$2,000,000 


$1,000,000 


$0 


($1,000,000) 


($2,000,000) 


San  Antonio 

I  I  Unadjusted 

Adjusted  +  standard  error 
-  Adjusted 

Adjusted  -  standard  error 


I  I  I  I  I  I  I  I  I 
Jan  Apr  Jul  Oct  Jan  Apr  Jul  Oct  Jan  Apr 

88  88  88  88  89  89  89  89  90  90 


Quarter  (first  month) 


Fig.  6 — San  Antonio  Gainshare  Pool  (Including  Air  Force  Share): 
A4justed  vs.  Unadjusted 
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Sacramento’s  labor  cost  under  PACER  SHARE  was  not  statistically 
different  from  that  at  baseline.  Although  that  comparison  showed  no 
statistically  significant  cost  savings  at  Sacramento  relative  to  its 
baseline,  the  comparison  omitted  a  key  possibility:  cost  could  have 
increased  at  other  ALCs  even  though  Sacramento’s  was  unchanged, 
in  which  case  Sacramento  might  have  achieved  comparative  cost 
savings.  The  results,  however,  did  not  support  that  possibility.  We 
found  that  Sacramento’s  cost  experience  did  not  differ  statistically 
from  that  of  other  ALCs.  This  left  us  with  an  apparent  paradox.  We 
found  no  statistically  significant  cost  savings,  but  Sacramento  had 
paid  gainshares  in  five-of-six  consecutive  quarters.  Our  gainshare 
computations  help  resolve  the  paradox.  Although  Sacramento  paid 
gainshares,  the  amounts  fell  within  the  range  expected  from  its  pre- 
PACER  SHARE  cost  behavior.  The  same  point  holds  even  when  we 
adjust  the  gainshares  for  change  in  output  level,  an  adjustment  work¬ 
ing  in  favor  of  finding  cost  savings.  We  therefore  conclude  that 
Sacramento  displays  a  tendency  toward  cost  savings  under  PACER 
SHARE  relative  to  its  baseline,  but  more,  similar  evidence  must  ac¬ 
cumulate  before  the  tendency  can  attain  statistical  significance.  At 
the  same  time,  a  parallel  tendency  characterizes  other  ALCs,  thereby 
reducing  the  prospect  for  attaining  comparative  cost  savings  under 
PACER  SHARE. 


7.  CONCLUSIONS 


The  goals  of  the  PACER  SHARE  Productivity  and  Personnel  Manage¬ 
ment  Demonstration  are  as  follows; 

•  Increase  organizational  productivity  by  improving  incentives  and 
training  that  will  help  employees  work  more  effectively  and  en¬ 
courage  them  to  originate  ideas  on  improving  efficiency. 

•  Increase  organizational  flexibility  in  making  job  assignments  and 
dealing  with  fluctuations  in  workload. 

•  Enrich  the  quality  of  work  life  by  creating  a  work  environment  in 
which  individual  and  organizational  goals  are  compatible,  opportu¬ 
nities  for  individuals  to  work  on  a  variety  of  jobs  are  realized,  and 
training  opportunities  are  expanded. 

•  Preserve  or  improve  the  quality  and  timeliness  of  work  through 
quality  circles,  team  building,  and  statistical  process  control. 

Are  these  goals  being  achieved?  In  this  section,  we  briefly  review  the 
findings  of  our  evaluation  through  the  second  year  of  the  demon¬ 
stration. 

PRODUCTIVITY 

Our  analysis  reveals  no  statistically  significant  cost  savings  when 
Sacramento’s  experience  during  the  demonstration  period  is  com¬ 
pared  with  its  baseline  experience,  or  when  Sacramento’s  overall  ex¬ 
perience  is  compared  with  that  of  other  ALCs.  Although  finding  no 
significant  cost  savings,  we  discovered  that  PACER  SHARE’S 
gainsharing  formula  underestimated  what  unit  cost  would  have  been 
in  view  of  Sacramento’s  lower  workload  during  the  demonstration 
period,  and  hence  has  underestimated  cost  savings  for  gainshare 
computation.  Had  an  adjustment  been  made,  larger  gainshares 
would  have  been  available  to  strengthen  the  feedback  effect  of  those 
payments  on  productivity.  Our  approach  could  contribute  toward 
modifying  the  gainshare  formula  to  ac^ust  for  changes  in  output  level. 
In  any  event,  the  actual  payment  of  gainshares  plus  the  possibility 
such  payments  could  have  been  higher,  sustains  the  prospect  of 
statistically  significant  cost  savings  relative  to  Sacramento’s  baseline. 
Yet  because  other  ALCs  also  have  a  tendency  for  cost  savings  relative 
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to  their  baseline,  it  remains  to  be  seen  whether  Sacramento  can 
attain  cost  savings  compared  with  them. 

ORGANIZATIONAL  FLEXIBILITY 

The  demonstration’s  success  at  promoting  organizational  flexibility 
and  quality  of  work  life  was  measured  through  an  employee  attitude 
survey  and  personnel  system  data.  Although  we  separate  the  survey 
and  personnel  system  measures  here  according  to  which  goal  they 
seem  most  clearly  associated  with,  it  should  be  kept  in  mind  that  or¬ 
ganizational  flexibility  and  quality  of  work  life  are  related.  For  ex¬ 
ample,  increases  in  earning  potential  brought  about  through  pay 
banding  (organizational  flexibility)  can  improve  the  perceived  quality 
of  work  life.  For  that  matter,  generally  speaking  the  four  goals  are 
interrelated,  so  many  of  our  measures  have  implications  for  more 
than  one  goal. 

Measures  related  to  organizational  flexibility  generally  exhibited  en- 
coura^ng  results,  although  the  pattern  was  not  universal.  Our  em¬ 
phasis  is  on  the  change  at  Sacramento  during  the  first  two  years  of 
the  demonstration  relative  to  the  change  in  DS  at  the  four  other  ALCs 
combined.  As  intended,  the  incidence  of  multiple  skill  training  in¬ 
creased  at  Sacramento  relative  to  the  comparison  sites.  Also  as  in¬ 
tended,  earnings  increased  for  nonsupervisory  employees  without  an 
increase  in  the  overall  wage  bill.  At  the  same  time,  there  was  no  evi¬ 
dence  of  pay  inversion  between  supervisory  and  nonsupervisory  posi¬ 
tions.  Crossovers  from  white-  to  blue-collar  positions  did  not  increase, 
although  higher  blue-  to  white-collar  changes  were  found  due  to  a  re¬ 
classification  action.  In  the  main,  however,  crossovers  remained  in¬ 
frequent.  Moreover,  the  percentage  of  career  employees  increased  at 
Sacramento  relative  to  the  comparison  sites,  as  the  total  work  force 
was  decreasing  in  size.^  Supervisors’  perceptions  of  support  from  the 
Personnel  Office,  the  fairness  of  their  grading  criteria,  and  staffing 
flexibility  improved,  as  did  work  force  perceptions  of  training  oppor¬ 
tunities,  job  competition  fairness,  and  gainsharing  of  organizational 
cost  savings.  On  the  negative  side,  the  percentage  of  supervisors 
within  DS  and  their  distribution  across  divisions  remained  the  same. 
Although  this  indicates  that  supervisory  positions  were  not  prolifer¬ 
ated,  it  also  suggests  that  the  flexibility  sought  through  changes  in 
supervisory  grading  criteria  had  not  yet  been  fully  realized.  Also, 
attitudes  toward  pay  and  promotion  under  revised  base  pay 


iThis  was  partly  attributable  to  holding  off  on  hiring  as  positions  were  vacated  by 
attrition. 
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determination  worsened.  Changes  in  other  attitudes  related  to 
flexibility — ^blue-  versus  white-collar  pay  equity,  staffing  needs,  job 
classification  satisfaction,  and  career  opportunities — were  not 
significantly  different  from  those  at  the  comparison  ALCs. 

QUALITY  OF  WORK  LIFE 

Individual  survey  questions  assessing  quality  of  work  life  were 
grouped  into  broadly  based  scales,  which  in  turn  were  classified  into 
several  broad  dimensions  (through  factor  analytic  methods).  The  di¬ 
mensions  and  scales  are  reviewed  in  Table  39. 

Measures  of  satisfaction  with  supervision  and  co-worker  interactions 
(the  first  area),  overall  work  satisfaction  (the  second  area),  and  mis¬ 
cellaneous  work  environment  perceptions  (the  last  area)  showed  sub¬ 
stantial  improvement  at  Sacramento  during  the  first  two  years  of 
PACER  SHARE.  Eight  of  the  scales  showed  differential  change,  and 
another  (reconsideration/redress)  showed  marginal  significance.  Only 
job  satisfaction,  intent  to  turn  over,  and  organizational  climate 
showed  changes  similar  to  those  at  the  comparison  sites.  However, 
the  pattern  was  quite  different  in  those  areas  concerned  with  pay. 
General  pay  satisfection  and  satisfaction  '^h  the  link  between  per¬ 
formance  and  advancement  worsened  i  .ame  significantly  more 
negative)  at  Sacramento  during  the  first  two  years  of  PACER  SHARE 
relative  to  the  comparison  sites.  We  attribute  the  former  result  in 
part  to  a  greater  increase  in  the  cost  of  living  in  the  Sacramento  area 
as  compared  with  the  areas  surrounding  the  other  ALCs.  However, 
some  overall  pay  dissatisfaction  and  dissatisfaction  with  the  link  be¬ 
tween  performance  and  advancement  appear  attributable  to  concerns 
about  salary  advancement  through  the  pay  bands  and  about  promo¬ 
tions  under  PACER  SHARE. 

The  pattern  of  results  was  even  stronger  when  a  statistical  control  for 
the  perceived  link  between  pay  and  performance  was  provided.  When 
we  look  at  the  nonfinancial  aspects  of  these  scales,  we  find  that  every 
scale  in  the  satisfaction  with  supervision  and  co-worker  interactions, 
overall  work  satisfaction,  and  miscellaneous  work  environment  per¬ 
ception  areas  improved  substantially  and  significantly  at  Sacramento 
relative  to  the  change  at  the  comparison  sites.  These  findings  not 
only  suggest  improvement  in  nonfinancial  aspects  of  quality  of  work 
life,  but  raise  the  hypothesis  that  to  the  extent  gainshares  can  be  paid 
and  concerns  about  advancement  through  the  pay  bands  addressed  in 
year  three,  we  may  observe  the  larger  pattern  of  change  in  the  full, 
unadjusted  attitude  results. 


Table  39 

Major  Survey  Areas 
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SATISFACTION  WITH  SUPERVISION  AND  CO-WORKER  INTERACTIONS 

General  supervision  and  direction,  group  functioning,  open  group  process,  satis¬ 
faction  with  supervision  and  work  unit 

OVERALL  WORK  SATISFACTION 

Intrinsic  work  satisfaction,  job  satisfaction,  intent  to  turn  over 
GENERAL  PAY  SATISFACTION 
External  equity,  pay  satisfaction 
REWARD  SYSTEM  SATISFACTION 

Pay  as  a  motivator,  pay-performance  link,  promotion  satisfaction 
MISCELLANEOUS  WORK  ENVIRONMENT  PERCEPTIONS 

Organizational  climate,  control  over  work,  reconsideration  and  redress,  training 
opportunities,  organizational  involvement 


Finally,  to  the  extent  that  PACER  SHARE  achieves  its  goal  of  im¬ 
proving  the  quality  of  work  life,  we  would  expect  turnover  to  decrease. 
At  baseline,  total  turnover  was  nearly  15  percent  at  Sacramento — 
significantly  higher  than  for  the  comparison  group — representing 
separations  of  nearly  11  percent  and  internal  transfers  (migration)  to 
other  directorates  of  just  under  4  percent,  both  significantly  higher 
than  for  the  comparison  ALCs.  During  year  two  of  PACER  SHARE, 
as  compared  with  the  other  ALCs,  Sacramento  showed  declines  in 
turnover.  Separations  decreased  comparably  at  Sacramento  and  the 
other  ALCs,  whereas  the  decline  in  internal  transfers  and  total 
turnover  was  significantly  greater  at  Sacramento.  As  a  result,  the 
year-two  turnover  rates  were  similar  for  Sacramento  and  the  compar¬ 
ison  ALCs.  Since  Sacramento  had  greater  turnover  at  baseline  (and 
previously),  the  change  is  consistent  with  an  improvement  in  quality 
of  work  life. 

WORK  QUALITY  AND  TIMELINESS 

Work  quality  began  at  a  superior  level  at  Sacramento,  and  for  error 
rates  it  showed  little  change  throughout  year  two  of  the  demonstra¬ 
tion.  Of  eight  error  rate  measures,  three  showed  improvement  at 
Sacramento  during  the  second  year  of  PACER  SHARE,  three  showed 
no  change,  and  two  worsened.  In  contrast,  measures  concerning  re¬ 
ceiving  timeliness  and  shipping  support  deteriorated  at  Sacramento 
relative  to  the  comparison  group.  This  may  be  at  least  partially  at- 
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tributable  to  the  implementation  of  the  Automated  Warehouse 
System  at  Sacreimento  and  to  support  for  the  F-15  program. 

A  number  of  attitude  questions  relevant  to  the  foregoing  discussion 
were  added  to  the  survey  at  year  one.  The  year-two  results  show  sig¬ 
nificant  increases  in  perceived  information  exchange  in  accomplishing 
work,  usefulness  of  quality  circle  participation,  and  emphasis  on 
team-building  concepts  in  day-to-day  operations.  Many  of  the 
changes — especially  those  for  team  building — were  large  and  highly 
significant.  The  changes  are  consistent  both  with  better  work  quality 
and  improved  quality  of  work  life. 

A  FINAL  NOTE 

Although  the  changes  in  federal  civil  service  practices  required  to 
implement  PACER  SHARE  were  in  place  at  its  outset,  it  must  be  rec¬ 
ognized  that  true  implementation  must  unfold  over  time.  For  exam¬ 
ple,  DS  employees  will  have  to  be  provided  training  to  take  advantage 
of  increased  personnel  system  flexibility  in  meeting  changing  work¬ 
loads,  DOC  hires  will  have  to  occur  over  time  as  the  need  arises,  and 
so  forth.  As  a  result,  in  future  evaluations  a  broader  range  of  mea¬ 
sures  will  become  available  for  analysis  and  current  measures  will 
become  more  meaningful.  It  is  important  to  wait  for  these  additional 
measures  and  to  observe  the  longer-term  results  of  PACER  SHARE 
before  drawing  firm  conclusions  concerning  its  effectiveness.  None¬ 
theless,  although  the  year-two  results  do  not  provide  evidence  of 
significant  cost  savings,  they  offer  encouragement  that  PACER 
SHARE  may  be  beginning  to  achieve  its  desired  objectives  in  other 
areas. 


Appendix  A 

PACER  SHARE  VS.  OTHER  OPM 
DEMONSTRATION  PROJECTS 


PACER  SHARE  is  one  of  several  demonstration  projects  being  con¬ 
ducted  under  the  authority  of  Title  VI  of  the  Civil  Service  Reform  Act. 
These  projects  test  innovations  in  public  personnel  management  by 
permitting  waivers  of  current  laws  and  regulations,  such  as  those  de¬ 
signed  to  improve  productivity  and  employee  performance  by  making 
the  federal  personnel  system  more  flexible  and  responsive.  Three  of 
those  demonstrations  began  before  PACER  SHARE: 

•  Integrated  Approach  to  Pay,  Performance  Appraisal,  and  Position 
Classification  for  More  Effective  Operation  of  Government 
Organizations  (Department  of  the  Navy) 

•  Alternative  Personnel  Management  System  (National  Institute  of 
Standards  and  Technology) 

•  Airway  Science  Curriculum  (Federal  Aviation  Administration) 

The  first  of  these  demonstrations  is  being  conducted  at  the  Naval 
Ocean  Systems  Center  in  San  Diego  and  the  Naval  Weapons  Center 
at  China  Lake.  Its  purpose  is  to  demonstrate  whether  the  effective¬ 
ness  of  federal  laboratories  can  be  enhanced  by  allowing  management 
greater  control  over  personnel  functions  and  expanding  the  opportu¬ 
nities  available  to  employees.  Like  PACER  SHARE,  the  project  ex¬ 
amines  the  benefits  of  a  simplified  classification  system  and  pay 
banding.  However,  it  does  not  emphasize  organizational  productivity, 
it  retains  performance  ratings,  and  it  provides  merit  pay  to  reward 
individual  performance  rather  than  gainsharing  to  reward  collective 
performance.  Also,  its  participants  are  largely  white  collar. 

The  demonstration  by  the  National  Institute  of  Standards  and 
Technology  has  much  in  common  with  the  Navy  project.  Its  goals  are 
to  simplify  the  classification  process,  make  it  more  understandable, 
and  place  more  decisionmaking  authority  with  line  managers.  It  es¬ 
tablishes  pay  banding  and  links  salaries  to  individual  performance. 
It  differs  from  the  Navy  demonstration  by  testing  such  innovations  as 
sabbaticals  and  compensation  comparability  with  the  private  sector. 
Participants  are  primarily  scientists  and  engineers. 
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The  FAA  demonstration  was  intended  to  develop  alternative  qualifi¬ 
cations  and  recruitment  sources  primarily  for  agency  technical  occu¬ 
pations.  It  thus  had  little  in  common  with  PACER  SHARE.  It  was 
conducted  between  1987  and  1991. 

Since  PACER  SHARE  began,  0PM  has  approved  two  more  demon¬ 
stration  projects.  One  is  another  FAA  demonstration,  this  one  testing 
retention  allowances  covering  difllcult-to-staff  positions  at  air  traffic 
control  facilties  in  the  Chicago,  Los  Angeles,  New  York,  and  Oakland 
areas.  The  other  is  a  test  of  skill-based  pay  by  the  Defense  Logistics 
Agency  at  its  Ogden,  Utah,  depot.  A  third  personnel  management 
demonstration  was  legislated  by  Congress.  It  provides  lump-sum  re¬ 
location  bonuses  and  retention  allowances  to  alleviate  severe  recruit¬ 
ment  and  retention  problems  at  the  FBI  in  New  York  City. 


Appendix  B 

RESULTS  FOR  THE  SINGLE-INTERCEPT  MODEL 


This  appendix,  a  companion  to  the  cost  savings  analysis  in  Sec.  6, 
presents  regression  results,  hypothesis  tests,  and  gainsharing  compu¬ 
tations  based  on  the  pooled-data,  single-intercept  model.  Even 
though  specific  estimates  differ,  that  model  leads  to  the  same  conclu¬ 
sion  as  the  multiple-intercept  model  described  in  the  text — no  statis¬ 
tically  significant  PACER  SHARE  cost  savings  (despite  the  payment 
of  gainshares  at  Sacramento). 

Table  B.l  contains  labor  cost  regressions  for  Sacramento,  all  other 
ALCs,  and  other  ALCs  excluding  Oklahoma  City.  Relative  to  results 
for  single  ALCs  (Table  31),  the  results  inclusive  of  Oklahoma  City  are 
implausible,  so  Oklahoma  City  was  dropped  from  further  analysis. 
(Note  the  high  intercept  and  low  output  effect  at  baseline,  the  high 
time  trend  in  the  demonstration  period,  and  the  low  R-square.)  The 
results  for  pooled  data  including  only  Ogden,  San  Antonio,  and 
Warner-Robins  agree  fairly  well  with  the  separate  ALC  regressions, 
although  some  differences  occur.  The  pooled-data  intercept  is  low  at 
baseline  and  high  in  the  demonstration  period,  whereas  the  output 
effect  is  high  at  baseline  and  low  in  the  demonstration.  These  differ¬ 
ences  result  from  the  data  pooling;  the  ALCs  tend  to  have  different 
ranges  of  output  but  similar  ranges  of  cost,  and  when  the  data  are 
pooled  the  overall  cluster  of  points  has  a  different  shape  than  that  of 
the  individual  ALC  clusters.  That  causes  the  low  intercept  and  high 
output  effect  at  baseline  and,  because  the  level  of  output  decreases 
and  its  overall  range  across  ALCs  broadens,  the  high  intercept  and 
low  output  effect  in  the  demonstration  period;  see  Fig.  2,  Sec.  6,  for 
the  cluster  patterns.  It  is  because  of  these  intercept  and  output  effect 
differences  that  we  prefer  the  multiple-intercept  model  of  Sec.  6. 

Table  B.l  reveals  change  at  Sacramento  and  other  ALCs  from  base¬ 
line  to  the  demonstration  period.  For  each,  the  intercept  rose  and  the 
output  effect  became  smaller.  Sacramento’s  intercept  changed  from 
11.28  to  13.81  compared  with  10.93  to  13.16  for  other  ALCs; 
Sacramento’s  output  effect  fell  from  .36  to  .14  versus  a  drop  of  .39  to 
.19  for  other  ALCs.  There  was  some  divergence  in  time  trend,  how¬ 
ever.  Sacramento’s  trend  fell  from  -.0019  to  -.0026,  whereas  other 
ALCs’  trend  rose  from  -.0004  to  .0007.  In  contrast  to  these  changes 
from  period  to  period,  Sacramento  and  other  ALCs  have  similar  coef- 
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Table  B.l 


Labor  Cost  Regressions:  Sacramento  vs.  Comparison  Group 
(dependent  variable:  fn  labor  cost) 


Variable 

Sacramento 

All  Other 
ALCs 

All  Other  ALCs  Except 
Oklahoma  City 

BASELINE  PERIOD 

Intercept 

11.28 

14.60 

10.93 

(2.86) 

(.95) 

(1.06) 

Time 

-.0019 

-.0002 

-.0004 

(.0014) 

(.0006) 

(.0006) 

output 

.36 

.08 

.39 

(.24) 

(.08) 

(.09) 

DEMONSTRATION  PERIOD 

Intercept 

13.81 

12.29 

13.16 

(2.69) 

(.82) 

(.88) 

Time 

-.0026 

.0029 

.0007 

(.0019) 

(.0009) 

(.0010) 

in  output 

.14 

.26 

.19 

(.23) 

(.07) 

(.07) 

Adj.  R-square 

.77 

.18 

.31 

Standard  error  of 

estimate 

.06 

.09 

.08 

NOTE:  Standard  errors  are  in  parentheses. 


flcients  within  a  period,  although  again  with  some  difference  in  time 
trend.  During  the  baseline  period,  the  intercepts  are  11.28  versus 
10.93  and  the  output  effects  are  .36  versus  .39  for  Sacramento  and 
other  ALCs,  respectively.  Sacramento  has  a  more  pronounced  trend, 
-.0019,  nearly  five  times  the  other  ALCs’  trend  of  -.0004.  During  the 
demonstration  period  the  intercepts  are  13.81  and  13.16,  the  output 
effects  .14  and  .19,  and  time  trends  -.0026  and  .0007.  Finally,  al¬ 
though  Sacramento’s  and  other  ALCs’  coefficients  changed  between 
the  baseline  and  demonstration  periods,  since  Sacramento’s  coeffi¬ 
cients  tend  to  lie  within  a  standard  error  of  the  other  ALCs’  coeffi¬ 
cients  in  each  period,  we  will  be  unlikely  to  find  relative  cost  reduc¬ 
tion  at  Sacramento.  The  hypothesis  tests  reported  next  bear  this  out. 

RESULTS  OF  HYPOTHESES  TESTS 

H5T)othesis  test  results  are  shown  in  Table  B.2.  (See  Sec.  6  for  infor¬ 
mation  on  interpreting  table  entries.)  The  results  are  consistent  with 
the  following  statements:  (1)  Sacramento’s  intercept,  time  trend,  and 
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Table  B.2 

Hypothesis  Test  Results 


HI  hypotheses:  Sacramento  versus  other  ALCs  during  baseline  period. 

tn  Cost 

Intercept 

Time  Trend 

/n  Output 

Equal?  Reject 

Accept 

Accept 

Accept 

Probability  .0039 

.90 

.28 

.89 

H2  hypotheses:  Other  ALCs,  demonstration  versus  baseline  periods. 

in  Cost 

Intercept 

Time  Trend 

£n  Output 

Equal?  Reject 

Reject 

Accept 

Reject 

Probability  .0014 

.10 

.36 

.09 

H3  h3TDotheses:  Sacramento,  demonstration  versus  baseline  periods. 

/n  Cost 

Intercept 

Time  Trend 

/n  Output 

Equal?  Accept 

Accept 

Accept 

Accept 

Probability  .27 

.48 

.71 

.47 

H4  hypotheses:  Sacramento  versus  other  ALCs  during  demonstration 
period. 


Equal? 

Probability 

^n  Cost 
Reject 
.0001 

Intercept 

Accept 

.80 

Time  Trend 
Reject 
.09 

fn  Output 
Accept 
.80 

H5  hypotheses:  Change  at  Sacramento,  demonstration  versus  baseline 

periods,  versus  change  at  other  ALCs,  demonstration 

versus  baseline  periods. 

(n  Cost 

Intercept 

Time  Trend 

^n  Output 

Equal? 

Accept 

Accept 

Accept 

Accept 

Probability 

.80 

.94 

.44 

.94 

output  effect,  when  taken  individually,  are  the  same  as  other  ALCs’ 
at  baseline.  However,  when  the  coefficients  are  viewed  as  a  group, 
Sacramento  differs  statistically  from  other  ALCs.  This  occurs  be¬ 
cause  the  sets  of  coefficients  are  more  precisely  measured  than  are 
single  coefficients.  Still,  the  results  suggest  that  Sacramento  and  the 
other  ALCs  are  comparable  at  baseline,  providing  a  reasonable  foun¬ 
dation  for  the  further  hypothesis  tests.  (2)  Other  ALCs’  cost  differs 
between  the  baseline  and  demonstration  periods.  The  factors  driving 
this  difference  are  the  change  in  intercept  and  output  coefficients.  As 
we  saw,  the  demonstration  period  intercept  is  larger  than  at  baseline 
and  the  output  effect  is  smaller.  The  gainsharing  computations  below 
imply  that,  controlling  for  output  leve’ ,  labor  cost  fell  from  baseline  to 
demonstration  period.  (3)  Sacramento’s  intercept  and  output  effect 
change  the  same  way  between  the  baseline  and  demonstration  peri¬ 
ods  but  labor  cost  does  not  differ  significantly.  While  true  based  on 
our  test,  the  result  reflects  the  smaller  precision  of  Sacramento’s  es¬ 
timates  due  to  its  smaller  sample  size  than  the  other  ALCs’  pooled- 
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data  sample.  Yet  as  stated  above,  Sacramento’s  intercept  and  output 
effects  change  by  virtually  the  same  amount  as  those  of  other  ALCs. 
(If  we  could  draw  upon  other  ALCs’  pattern  in  judging  Sacramento’s 
change  rather  than  keeping  it  apart  as  a  contrast,  we  might  well  con¬ 
clude  that  Sacramento’s  change  was  statistically  significant  too.)  And 
as  true  for  the  other  ALCs,  the  gainsharing  computations  for 
Sacramento  indicate  cost  improvement  over  baseline.  (4)  Sacra¬ 
mento’s  cost  differs  from  other  ALCs’  cost  in  the  demonstration 
period,  a  result  driven  by  the  difference  in  time  trend.  (5)  The  change 
in  Sacramento’s  cost  from  baseline  to  the  demonstration  period  is  the 
same  as  the  change  in  other  ALCs’  cost.  Overall,  while  there  are 
some  differences  from  the  multiple-intercept  model  results,  the  main 
conclusion  remains  the  same.  Relative  to  its  own  past  performance 
and  the  performance  of  other  ALCs,  Sacramento  has  not  attained  sta¬ 
tistically  significant  cost  savings  under  PACER  SHARE. 

GAINSHARING 

The  reader  is  referred  to  Sec.  6  for  discussion  of  the  gainsharing  for¬ 
mula  and  the  adjustment  we  make  to  unit  cost.  Here  we  focus  on 
ALCs  other  than  Sacramento.  Sacramento’s  gainsharing  computa¬ 
tions  do  not  differ  from  those  reported  in  Sec.  6  because  its  model  has 
not  changed.  But  other  ALCs’  gainshares  change  because  now  they 
are  based  on  a  single-intercept  model  rather  than  the  multiple-inter¬ 
cept  model.  In  the  following  we  first  test  for  independence  of  unit  la¬ 
bor  cost  from  output  and  then  present  gainshare  computations  that 
are  both  unadjusted  and  adjusted  for  output  level.  We  found  before 
that  the  adjustment  results  in  higher  gainshares  at  both  Sacramento 
and  other  ALCs.  We  find  the  same  result  for  other  ALCs  now. 

As  before,  the  test  rejects  independence  of  other  ALCs’  unit  cost  from 
output  (Table  B.3).  The  regression  results  for  Sacramento  and  the 
other  ALCs  both  imply  that  a  1  percent  increase  in  output  is  associ¬ 
ated  with  a  .6  percent  drop  in  the  unit  cost  in  baseline  period  and  a  .8 
percent  drop  during  the  demonstration  period.  The  tests  are  highly 
significant  in  all  cases. 

Table  B.4  and  Fig.  B.l  show  how  the  dependence  of  unit  cost  on  out¬ 
put  affects  the  gainshare  computation  for  the  other  ALCs.  The  tables 
are  constructed  with  “would  have”  unit  cost  equal  to  average  baseline 
unit  cost  and,  alternatively,  adjusted  baseline  unit  cost.  (The  figures 
are  illustrative,  not  official.)  Briefly,  gainshare  =  (“would  have”  unit 
cost  -  current  unit  cost)  x  current  output.  In  addition,  we  present 
adjusted  “would  have”  unit  costs  for  a  standard  error  above  and  below 
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Table  B.3 


Tests  for  Independence  of  Unit  Cost  from  Output 


Site 

aj-l 

t-value 

Reject  Independence 

(82  -  1)  =  0? 

BASELINE 

Sacramento 

-.64 

(.23) 

-2.8 

yes 

Other  ALCa  -.61 

(.09) 

DEMONSTRATION 

-6.8 

yes 

Sacramento 

-.86 

(.24) 

-3.6 

yes 

Other  ALCs 

-.81 

(.07) 

-11.6 

yes 

NOTE;  A]]  t-values  exceed  the  .01  significance  level  of  2.4. 
Estimates  of  82  come  from  Table  B.l  for  other  ALCs  and  Table  31 
for  Sacramento.  Standard  errors  are  in  parentheses. 


the  regression  prediction,  and  show  the  associated  gainshare  compu¬ 
tation  for  these  values.  As  in  Sec.  6,  the  ac^usted  gainshares  exceed 
the  unadjusted,  providing  further  support  for  the  findings  and  con¬ 
clusions  drawn  there. 

GAINSHARE  PAYMENTS  AT  SACRAMENTO 

Table  B.5  shows  the  unit  cost  savings  and  associated  gainshares  de¬ 
clared  by  Sacramento  since  the  initiation  of  the  demonstration. 


Table  B.4 
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Fig.  B.l — Other  ALCs’  Gainshare  Pool  (Including  Air  Force  Share): 
Unadjusted  vs.  Adjusted  via  Single-Intercept  Model 


Table  B.6 

Gainshare  Payments  by  Sacramento 


Quarter  and  CY 

Unit  Cost  Savings 

Employee  Gainshare 

1  qtr  CY88 

0 

0 

2  qtr 

0 

0 

3  qtr 

0 

$128.09“ 

4  qtr 

0 

0 

1  qtr  CY89 

$527,018 

$163.64 

2  qtr 

$1,003,614 

$312.31 

3  qtr 

$84,694 

$26.57 

4  qtr 

0 

0 

1  qtr  CY90 

$187,352 

$59.49 

2  qtr 

$312,501 

$102.08 

“Performance  Award. 


Appendix  C 

LABOR  COST  AND  OUTPUT  DATA  BY  ALC 


This  appendix  lists  the  data  used  in  the  year-two  analysis  of  cost  sav¬ 
ings.  The  labor  cost  data  are  for  paid  labor,  including  straight-time 
hours  of  work  and  overtime,  as  well  as  paid  time  away  from  work 
such  as  sick  leave  and  vacation.  The  labor  cost  data  (Table  C.l)  have 
been  inflated/deflated  to  1989  dollars;  the  factor  for  each  month  is 
also  listed.  These  constant-dollar  labor  costs  were  used  in  the  empiri¬ 
cal  analysis.  The  output  data  (Table  C.2)  indicate  the  total  number  of 
issues  and  receipts,  including  receipts  from  off  base,  on-base  receipts 
from  maintenance,  on-base  receipts  not  from  maintenance,  issues  off 
base,  on-base  issues  to  maintenance,  on-base  issues  not  to  mainte¬ 
nance,  and  on-base  issues  to  disposal. 
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Table  C.l 
Labor  Cost  Data,  $ 


Date 

Oklahoma 

City 

Ogden 

Oct-84 

5244860 

7029433 

Nov-84 

5533141 

6885598 

Dec-84 

5199018 

6261187 

Jan-85 

5419407 

7155645 

Feb-85 

4578549 

6341882 

Mar-85 

5298545 

6406640 

Apr-85 

5535439 

5602601 

May-85 

5839536 

5888023 

Jun-85 

5210097 

5268121 

Jul-85 

5205752 

5920013 

Aug-85 

6352618 

5861970 

Sep-85 

5039336 

5501429 

Oct-85 

5743991 

5327348 

Nov-85 

5346757 

5556477 

Dec-85 

5499791 

5102608 

Jan-86 

5394951 

5533027 

Feb-86 

5656453 

4834580 

Mar-86 

5541549 

5038788 

Apr-86 

5267064 

7621380 

May-86 

5743941 

7810091 

Jun-86 

5624078 

5944605 

Jul-86 

5866395 

6427723 

Aug-86 

5753537 

6065523 

Sep-86 

6210767 

7092318 

Oct-86 

6215824 

6625067 

Nov-86 

5919390 

5307166 

Dec-86 

5925348 

5982901 

Jan-87 

5850483 

6207345 

Feb-87 

5366406 

6449998 

Mar-87 

5876587 

6372770 

Apr-87 

5688664 

6499864 

May-87 

5796076 

6189834 

Jun-87 

5254845 

5644357 

Jul-87 

5703137 

6784775 

Aug-87 

5143135 

6171845 

Sep-87 

5038226 

6940080 

Oct-87 

5269141 

5368642 

San 

Antonio 

Sacramento 

Warner- 

Robins 

5661520 

5713648 

6723636 

5026940 

5535273 

6341749 

5382266 

5216358 

6187527 

5875764 

5975360 

6833616 

5048219 

5100813 

6027830 

5325799 

5320845 

6322820 

5648382 

5459784 

6516264 

5865889 

5849814 

6798622 

5391965 

5050946 

5877516 

5994162 

5870632 

6864724 

5864320 

5721519 

6546292 

5847881 

5639487 

6252255 

5778382 

6030701 

7010375 

5845389 

5710500 

6448094 

6437309 

5878821 

6806948 

6410201 

6191439 

7463399 

5560711 

5376796 

5850268 

5978151 

6378870 

6097311 

5952227 

6658018 

5941970 

5923174 

6740416 

5818595 

5588127 

6543279 

6190608 

6050263 

6765993 

5870730 

5267615 

6604762 

6058399 

5723197 

6820819 

6090018 

5845139 

6951956 

5311875 

5263425 

5968173 

6178417 

5532027 

6560552 

6084184 

5674308 

6791546 

5486204 

5064245 

5885022 

6015959 

5450651 

6308071 

5866206 

6694918 

5458890 

4944680 

6242759 

5854667 

5367419 

6308483 

5989212 

5326850 

6469377 

5479589 

4804077 

5978369 

5945771 

5349091 

6374351 

5636967 

5099681 

5871287 

Inflator 

1.2070 

1.2060 

1.2050 

1.2040 

1.2030 

1.2020 

1.2010 

1.2000 

1.1990 

1.1980 

1.1970 

1.1960 

1.1950 

1.1940 

1.1930 

1.1920 

1.1910 

1.1900 

1.1849 

1.1798 

1.1748 

1.1697 

1.1646 

1.1595 

1.1544 

1.1493 

1.1443 

1.1392 

1.1341 

1.1290 

1.1212 

1.1133 

1.1055 

1.0977 

1.0898 

1.0820 

1.0742 
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Table  C.l — continued 


Date 

Oklahoma 

City 

Ogden 

San 

Antonio 

Sacramento 

Warner- 

Robins 

Inflator 

Nov-87 

4664314 

5069099 

5487315 

4588790 

5764885 

1.0663 

Dec-87 

5465535 

4657181 

5803540 

5141635 

6308487 

1.0585 

Jan-88 

4447451 

6048297 

5414078 

4734639 

5733206 

1.0507 

Feb-88 

4629715 

5123228 

4732771 

4536616 

5339799 

1.0428 

Mar-88 

5136778 

4796036 

5364016 

4921734 

5574137 

1.0350 

Apr-88 

5113896 

4551735 

4977614 

4312676 

5238557 

1.0321 

May-88 

4255265 

6452257 

5055607 

4630146 

6221299 

1.0292 

Jun-88 

4820184 

5100554 

5108571 

4810555 

5623493 

1.0263 

Jul-88 

4387240 

4896703 

4869956 

4453182 

5325920 

1.0233 

Aug-88 

5835813 

6239153 

5311183 

4850413 

5486684 

1.0204 

Sep-88 

4726547 

6228533 

5225453 

4855541 

5646444 

1.0175 

Oct-88 

5684011 

5956193 

5224611 

4611890 

5330494 

1.0146 

Nov-88 

5760061 

5181735 

4793663 

4591943 

5563710 

1.0117 

Dec-88 

5909694 

5375523 

5292628 

4589189 

5795494 

1.0088 

Jan-89 

6058367 

5558274 

5879274 

4530881 

6188665 

1.0058 

Feb-89 

5351256 

5202021 

5541660 

4219697 

5311141 

1.0029 

Mar-89 

6850949 

5720112 

6361490 

4861290 

6597952 

1.0000 

Apr-89 

5725277 

5548740 

5296000 

4125238 

5226815 

0.9971 

May-89 

6397665 

6077767 

5980661 

4505136 

6032235 

0.9942 

Jun-89 

6144100 

6041959 

5676546 

4718052 

5733509 

0.9913 

Jul-89 

5902236 

5887794 

5268305 

4031667 

5673460 

0.9883 

Aug-89 

6352834 

5637906 

5686543 

4561741 

5993545 

0.9854 

Sep-89 

6123985 

5637927 

54X8610 

4132758 

5506369 

0.9825 

Oct-89 

6009784 

4893342 

5494079 

4083274 

5692055 

0.9796 

Nov-89 

5985122 

5005882 

56X6294 

4445733 

6106716 

0.9767 

Dec-89 

5726608 

4918846 

5304238 

4435230 

5385022 

0.9738 

Jan-90 

6198821 

5423976 

5880011 

4582994 

5962567 

0.9708 

Feb-90 

5772377 

4938735 

5082882 

4173104 

5130167 

0.9679 

Mar-90 

6092487 

5457396 

5567865 

4375011 

5932424 

0.9650 

Apr-90 

5926099 

5454026 

5444690 

4449658 

5283836 

0.9621 

May-90 

6259806 

5756368 

5327673 

4758615 

5726832 

0.9592 

Jun-90 

5707033 

5503390 

5404862 

4187181 

5111468 

0.9563 
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Table  C.2 

Output  Data:  Issues  and  Receipts 


Date 

Oklahoma 

City 

Ogden 

San 

Antonio 

Sacramento 

Warner- 

Robins 

Oct-84 

189539 

173156 

180579 

176157 

196547 

Nov-84 

181642 

172683 

162343 

157354 

182155 

Dec-84 

163396 

155024 

155481 

151960 

179442 

Jan-85 

189386 

170798 

156780 

163575 

194008 

Feb-85 

180063 

159354 

158043 

159796 

186101 

Mar-85 

227940 

177137 

187852 

172913 

213005 

Apr-85 

200467 

181335 

191274 

172440 

207790 

May-85 

209955 

179540 

180511 

166943 

207658 

Jun-85 

190665 

144648 

160946 

156359 

176135 

Jul-85 

202900 

157313 

178005 

164842 

182299 

Aug-85 

216464 

141557 

181202 

169618 

204432 

Sep-85 

194839 

182033 

164999 

156897 

184359 

Oct-85 

205359 

192337 

191417 

165766 

205814 

Nov-85 

193661 

173590 

175067 

151966 

183958 

Dec-85 

193670 

171793 

168593 

149736 

181753 

Jan-86 

201055 

183950 

174727 

156724 

189047 

Feb-86 

188481 

168557 

205647 

147480 

182485 

Mar-86 

207833 

187154 

190502 

185279 

190652 

Apr-86 

205062 

193246 

189650 

185914 

188183 

May-86 

188697 

164767 

167986 

162619 

186008 

Jun-86 

190865 

171506 

168657 

169238 

175110 

Jul-86 

202900 

178613 

177630 

167895 

183019 

Aug-86 

216464 

156231 

179067 

162920 

191482 

Sep-86 

195109 

186227 

175027 

157792 

188351 

Oct-86 

201202 

162600 

189471 

170381 

187890 

Nov-86 

184674 

152795 

165297 

151202 

168354 

Dec-86 

185815 

157778 

164798 

155022 

168685 

Jan-87 

173668 

155106 

170231 

151373 

170670 

Feb-87 

185385 

170812 

169569 

159666 

180581 

Mar-87 

219388 

191469 

191485 

181899 

211191 

Apr-87 

211823 

177990 

191778 

169348 

203129 

May-87 

178203 

156354 

182334 

156223 

192059 

Jun-87 

193533 

166239 

184045 

155453 

186882 

Jul-87 

186806 

168517 

186833 

157572 

179566 

Aug-87 

192555 

157335 

181490 

156230 

182901 

Sep-87 

203842 

165798 

185649 

150811 

192426 

(  t 

i 

I 


i 
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Table  C.2 — continued 


Oklahoma  San  Warncr- 

Date  City  Ogden  Antonio  Sacramento  Robins 


Oct-87 

Nov-87 

Dec-87 

Jan-88 

Feb-88 

Mar-88 

Apr-88 

May-88 

Jun-88 

Jul-88 

Aug-88 

Sep-88 

Oct-88 

Nov-88 

Dec-88 

Jan-89 

Feb-89 

Mar-89 

Apr-89 

May-89 

Jun-89 

Jul-89 

Aug-89 

Sep-89 

Oct-89 

Nov-89 

Dec-89 

Jan-90 

Feb-90 

Mar-90 

Apr-90 

May-90 

Jun-90 


198503 

175003 

171594 

153500 

184626 

193147 

184451 

169868 

174023 

170780 

195203 

181163 

193292 

185059 

187536 

197825 

180893 

209133 

189480 

196483 

187925 

168171 

213041 

182522 

193619 

184390 

159441 

188001 

178877 

198730 

176313 

183509 

172681 


161152 

153074 

146166 

136468 

140821 

162598 

149607 

143975 

130356 

130510 

163583 

177609 

150311 

146430 

136276 

162090 

139019 

170032 

162513 

161948 

151732 

136212 

165474 

142777 

146134 

147965 

135170 

145957 

138897 

165469 

140471 

143565 

126022 


191638 

163139 

168732 

158926 

172694 

179352 

177723 

167425 

154471 

140624 

170520 

162742 

168609 

158963 

172014 

185537 

178623 

213344 

197575 

197410 

202113 

172633 

206687 

174307 

194768 

173174 

160871 

169402 

169325 

189379 

173609 

186320 

178056 


154393 

140744 

134379 

134638 

137272 

148318 

134998 

130126 

125972 

118811 

145740 

124665 

135182 

118002 

125429 

136427 

131744 

149125 

139455 

150743 

133475 

126966 

148025 

131665 

134527 

131639 

115089 

132488 

118090 

136699 

126365 

127627 

119828 


192099 

186152 

173587 

153559 

167961 

194059 

180907 

167354 

159296 

155016 

173938 

175020 

163687 

168847 

163925 

198965 

169957 

171626 

173530 

185493 

178586 

147455 

197321 

160555 

172599 

163303 

145083 

175984 

177151 

189439 

167895 

168855 

166915 
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